-- Logs begin at Thu 2022-07-21 20:01:35 UTC, end at Thu 2022-07-21 21:19:45 UTC. -- Jul 21 20:37:18.337483 dv-cinder-850687-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 21 20:37:18.346536 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 21 20:37:18.350239 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 21 20:37:18 2022] *** Jul 21 20:37:18.350239 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 21 20:37:18.350239 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 21 20:37:18.350239 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: nodename: dv-cinder-850687-1-iscsi Jul 21 20:37:18.350239 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: machine: x86_64 Jul 21 20:37:18.350239 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: clock source: unix Jul 21 20:37:18.350239 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: pcre jit disabled Jul 21 20:37:18.350239 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: detected number of CPU cores: 4 Jul 21 20:37:18.350694 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: current working directory: / Jul 21 20:37:18.350694 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: detected binary path: /usr/bin/uwsgi-core Jul 21 20:37:18.350694 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: your processes number limit is 47680 Jul 21 20:37:18.350694 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: your memory page size is 4096 bytes Jul 21 20:37:18.350694 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: detected max file descriptor number: 2048 Jul 21 20:37:18.350694 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: lock engine: pthread robust mutexes Jul 21 20:37:18.350694 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: thunder lock: enabled Jul 21 20:37:18.351026 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 21 20:37:18.351026 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 21 20:37:18.399337 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: Python main interpreter initialized at 0x55c6c4c2b9c0 Jul 21 20:37:18.399337 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: python threads support enabled Jul 21 20:37:18.399337 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: your server socket listen backlog is limited to 100 connections Jul 21 20:37:18.399337 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: your mercy for graceful operations on workers is 90 seconds Jul 21 20:37:18.399830 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: mapped 403077 bytes (393 KB) for 2 cores Jul 21 20:37:18.399830 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: *** Operational MODE: preforking *** Jul 21 20:37:18.400122 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: *** uWSGI is running in multiple interpreter mode *** Jul 21 20:37:18.400122 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: spawned uWSGI master process (pid: 87054) Jul 21 20:37:18.401038 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: spawned uWSGI worker 1 (pid: 87060, cores: 1) Jul 21 20:37:18.400328 dv-cinder-850687-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 21 20:37:18.402774 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: spawned uWSGI worker 2 (pid: 87061, cores: 1) Jul 21 20:37:18.402774 dv-cinder-850687-1-iscsi devstack@keystone.service[87054]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 21 20:37:23.996643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 20:37:23.996643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: from cryptography.utils import int_from_bytes Jul 21 20:37:23.997447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 20:37:23.997447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: from cryptography.utils import int_from_bytes Jul 21 20:37:24.004231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 20:37:24.004231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: from cryptography.utils import int_from_bytes Jul 21 20:37:24.004743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 20:37:24.004743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: from cryptography.utils import int_from_bytes Jul 21 20:37:24.046172 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 20:37:24.046354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 20:37:24.046544 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] command line args: [] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 20:37:24.046806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 20:37:24.046992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ================================================================================ {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 20:37:24.047238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] admin_token = **** {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.047480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] config_dir = [] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.047664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.047872 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] config_source = [] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.048046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] debug = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.048242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.048606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.048783 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.049030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.049287 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.049519 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] list_limit = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.049726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] log_config_append = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.049928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.050115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] log_dir = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.050304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] log_file = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.050563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.050817 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.051147 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.051321 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.051497 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.051571 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 20:37:24.051671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.051794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 20:37:24.051937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.052022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] command line args: [] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 20:37:24.052182 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.052298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 20:37:24.052442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ================================================================================ {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 20:37:24.052600 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.052768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] admin_token = **** {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.052850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.053077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] config_dir = [] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.053251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.053336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.053415 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.053511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] config_source = [] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.053651 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] debug = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.053755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.053917 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.054115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.054222 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.054332 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.054497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.054497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.054769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.054939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.055056 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] publish_errors = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.055215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] list_limit = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.055215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] log_config_append = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.055569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.056322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.056447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.056617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] log_dir = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.056729 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.056833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] log_file = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.057023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.057131 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.057244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.057357 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.057474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] standard_threads = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.057595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.057595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.057806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.057930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.058048 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.058151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.058271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.058379 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.058498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] use_journal = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.058611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.058738 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] use_json = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.058738 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] use_stderr = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.058955 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.059076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] use_syslog = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.059187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.059295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.059429 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.059548 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.059653 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.059921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.060042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.060042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.060258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.060432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.060542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.060542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.060733 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.060846 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.060951 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] publish_errors = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.061058 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.061181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.061291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.061410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.061526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.061634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.061736 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.061849 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.061849 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.062088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.062192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] standard_threads = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.062305 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.062537 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.062649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.062759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.062876 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.062983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.063088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.063199 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] use_journal = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.063199 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] use_json = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.063398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.063526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] use_stderr = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.063630 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.063774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] use_syslog = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.063774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 20:37:24.063973 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.064093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.064216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.064342 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.064455 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.064565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.064670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.064779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.064911 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.065017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.065120 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.065225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.065331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.065433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.065639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.065749 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.065871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.065973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.066078 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.066192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.066313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.066420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.066528 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.066650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.066765 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.066866 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.066985 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.067099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.067220 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.067338 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.067443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.067555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.067672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.067814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.067933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.068058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.068058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.068273 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.068392 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.068507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.068620 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.068727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.068841 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.068958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.069090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.069200 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.069306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.069413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.069541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.069671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.069775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.069880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.069992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.070099 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.070207 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.070324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.070438 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.070553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.070658 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.070760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.070873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.071109 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.071239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.071349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.071471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.071588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.071680 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.071889 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.072003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.072102 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.072204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.072320 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.072443 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.072443 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.072660 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.072766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.072913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.073053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.073189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.073338 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.073495 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.073643 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.073780 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.073929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.074071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.074206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.074332 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.074332 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.074534 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.074642 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.074814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.074949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.075057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.075174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.075283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.075283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.075476 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.075476 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.075810 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.075923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.076028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.076136 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.076136 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.076409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.076535 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.076535 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.076777 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.076893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.077005 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.077108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.077302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.077414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.077520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.077628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.077789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.077898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.078071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.078237 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.078396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.078507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.078615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.078724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.078724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.078999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.079121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.079235 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.079355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.079457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.079596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.079746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.079855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.079962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.080069 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.080174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.080301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.080417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.080536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.080650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.080753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.080860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.080984 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.081094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.081202 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.081322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.081450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.081572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.081691 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.081793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.081914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.082034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.082140 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.082254 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.082373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.082373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.082576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.082683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.082799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.082916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.083123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.083234 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.083343 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.083460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.083569 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.083682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.083861 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.083979 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.084098 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.084210 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.084318 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.084419 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.084536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.084656 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] auth.external = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.084789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.084898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.085003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.085110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.085219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.085322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.085444 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.085553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] auth.password = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.085663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.085776 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] auth.token = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.085879 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.085995 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.086106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.086221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.086330 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.086440 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.086567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.087220 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.087360 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.087488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.087607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.087757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.087865 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.088002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.088166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.088289 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] credential.caching = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.088410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.088521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.088631 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.088738 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.088846 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.088966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.089093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.089200 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.089318 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.089429 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.089541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.089665 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.089784 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.089912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.090028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] auth.external = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.090133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.090240 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.090368 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.090491 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.090600 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.090705 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.090815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.090933 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] auth.password = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.091052 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.091161 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] auth.token = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.091270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.091393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.091511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.091618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.091757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.091872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.091991 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] federation.caching = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.092101 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.092224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.092346 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.092468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.092579 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.092689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.092815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.092928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.093034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] credential.caching = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.093153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.093264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.093379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.093486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.093593 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.093702 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.093840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.093947 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.094063 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.094173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.094283 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.094400 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.094533 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.094658 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.094767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity.caching = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.094886 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.094995 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.095119 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.095250 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.095358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.095466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.095580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.095691 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.095820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.095945 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.096059 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.096177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.096283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.096390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.096498 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.096620 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.096732 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] federation.caching = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.096849 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.096974 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.097095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.097198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.097419 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.097530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.097718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.097830 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.097952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.098089 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.098209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.098320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.098429 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.098537 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.098644 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.098765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.098877 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.098994 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.099102 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.099215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.099331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.099455 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.099570 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.099679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity.caching = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.099912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.100032 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.100137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.100257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.100372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.100495 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.100602 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.100710 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.100819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.100939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.101048 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.101155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.101263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.101481 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.101591 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.101712 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.101824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.101936 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.102046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.102046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.102265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.102265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.103068 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.103186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.103294 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.103294 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.103518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.103635 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.103769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.103889 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.104018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.104129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.104484 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.104612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.104612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.104818 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.104930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.105042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.105150 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.105271 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.105384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.105384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.105586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.105712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.105831 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.105949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.106059 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.106173 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.106298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.106408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.106515 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.106635 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.106745 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.106852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.106958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.107076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.107189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.107313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.107420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.107528 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.107638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.107832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.107832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.108072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.108192 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.108296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.108544 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.108670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.108792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.108913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.109024 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.109133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.109237 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.109344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.109452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.109580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.109696 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.109805 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.109916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.110027 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.110137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.110262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.110390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.110512 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.110616 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.110733 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.110847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.110970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.111072 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.111172 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.111276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.111384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.111520 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.111640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.111765 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.111877 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.111987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.112096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.112224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.112333 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.112443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.112560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.112665 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.112665 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.112878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.112990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.113086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.113195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.113384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.113506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.113621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.113621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.113810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.113938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.114020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.114080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.114168 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.114168 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.114311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.114378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.114474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.114593 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.114696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.114786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.114878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.114974 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.115083 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.115191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.115322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.115429 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] resource.caching = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.115539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.115632 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.115740 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.115804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.115913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.115988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.116047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.116123 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.116240 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.116353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.116520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.116631 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.116745 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.116836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.116897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.116959 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.117039 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.117112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.117177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.117308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] role.caching = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.117388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.117553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] role.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.117683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.117798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.117875 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.118098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.118183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.118393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.118490 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.118645 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.118734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.118887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.118998 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.119148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.119256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.119362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.119502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.119606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.119782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.119866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.120021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.120098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.120256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.120368 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.120500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.120580 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.120748 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.120829 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.120992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.121070 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.121239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] resource.caching = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.121437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.121535 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.121668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.121747 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.121917 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.121996 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.122165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.122247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.122411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.122490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.122659 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.122738 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.122892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.122973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.123135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.123213 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.123380 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.123458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.123614 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.123693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] role.caching = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.123879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.123962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] role.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.124110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.124189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.124340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.124422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.124590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.124669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.124839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.124916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.125082 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.125161 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.125315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.125394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.125547 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.125626 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.125771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.125851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.126002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] token.caching = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.126080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.126236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.126316 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.126483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.126563 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.126714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.126793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.126966 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.127044 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.127209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.127288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.127452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.127628 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.127707 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.127902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.127994 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.128157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.128237 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.128402 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.128479 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.128638 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.128726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.128895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.128987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.129113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.129193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.129358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.129440 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.129604 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.129725 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.129960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.130047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.130047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.130499 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.130580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.130641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.connection = **** {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.130722 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.130804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.130877 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.133547 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.133719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.133808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.136303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.136403 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.136491 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.136573 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.138474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.138561 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.138625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.138685 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.138743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] token.caching = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.138801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.139058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.139141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.139249 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.139335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.139409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.139479 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.139560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.139635 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.139707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.139792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.139863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.139923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140005 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 20:37:24.140627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.connection = **** {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.140627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 20:37:24.141501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87061) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 20:37:24.166599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.166817 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.167033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.167679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.167933 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.168160 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.175579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.176734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.205429 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.205651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.206728 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.206947 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.353508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.355593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.405527 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.405770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.411649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.411905 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.473498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.473719 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.473812 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.473940 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.474028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.474159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.474272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.474370 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.474468 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.474574 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.474659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.474799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.501138 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.501364 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.501564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.501682 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.501761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.501836 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.501979 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.502083 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.502187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.502268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.502401 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.502495 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.502696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.502912 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.513926 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.514703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.526794 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.527003 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.527229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.527449 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.527595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.527704 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.527855 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.527933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.528083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.528166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.528314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.528536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.528758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.528968 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87061) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 20:37:24.545796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.546002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 20:37:24.546133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.546315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 20:37:24.546414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.546520 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 20:37:24.546635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.546771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.546894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 20:37:24.547036 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.547168 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.547251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.547325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.547508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.547614 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.547854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.559533 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.559845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.559941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.560118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.560224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.560409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.560552 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.560743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.568552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.568840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.572112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.572516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.572604 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.572840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.573088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.573206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.573491 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.573580 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.573776 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.573895 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.574040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.574171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.574291 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.574545 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.574674 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.574822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.574997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.575095 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.575433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.575598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.575778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.601935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.602242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.604622 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.604731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.604834 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.604964 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.610094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.610194 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.610369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.610470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.610660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.610745 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.617991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.618615 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.618928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.619078 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.623548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.623657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.623854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.623942 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.624169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.624262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.624425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.624509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.624693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.624826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.624964 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.625049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.625227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.625311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.625489 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.625575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.644135 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.644281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.644411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.644530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.648936 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.649117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.652633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.652920 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.653075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.653375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.657912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.658394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.661546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.662051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.665848 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.666145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.666433 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.666534 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.666719 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.666825 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.667002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.667135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.667269 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.667392 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.667598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.667913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.686829 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.687208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.687452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.687567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.687741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.687866 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.688141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.688408 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.695600 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.696340 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.697485 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.697754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.698046 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.698321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.700806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.701083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.701362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.701501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.701639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.701730 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.701914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.702001 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.702193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.702534 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.715294 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.715571 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.715874 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.715960 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.716158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.716257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.716431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.716514 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.716712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.716796 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.716983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.717068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.717267 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.717545 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.737114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.737599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.740407 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.740883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.744311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.744799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.745973 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.746412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.747593 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.747890 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.748055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.748180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.748334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.748622 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.755508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.756145 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.757171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.757755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.760451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.760733 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.761013 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.761102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.761293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.761392 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.761615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.761892 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.770424 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.770708 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.770981 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.771133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.771345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.771626 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.778826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.779100 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.779379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.779649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.779760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87061) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 20:37:24.779943 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.780031 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.780220 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.780317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.780487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.780571 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.780762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.780873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.781031 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.781127 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.781294 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.781437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.781566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.781651 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.781831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.781934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.782091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.782185 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.782353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.782450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.782630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.782892 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.783159 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87061) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 20:37:24.820933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c6c4c2b9c0 pid: 87060 (default app) Jul 21 20:37:24.822211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c6c4c2b9c0 pid: 87061 (default app) Jul 21 20:37:24.824533 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17759d07-1784-43ef-ba77-5fb6eade0f7e None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:24.824822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17759d07-1784-43ef-ba77-5fb6eade0f7e None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:24.825098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17759d07-1784-43ef-ba77-5fb6eade0f7e None None] PATH_INFO: `/v3` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:24.829273 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1/1] 10.222.0.60 () {50 vars in 791 bytes} [Thu Jul 21 20:37:24 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:32.944816 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1a780-9c91-4718-bddb-2bb3bdebf82f None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:32.945387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1a780-9c91-4718-bddb-2bb3bdebf82f None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:32.945387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1a780-9c91-4718-bddb-2bb3bdebf82f None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:32.950289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1/2] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:32.955677 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd00673-e618-4c85-9212-703d0069ca4e None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:32.956105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd00673-e618-4c85-9212-703d0069ca4e None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:32.956263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd00673-e618-4c85-9212-703d0069ca4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:33.028082 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: INFO dbcounter [None req-ccd00673-e618-4c85-9212-703d0069ca4e None None] Registered counter for database keystone Jul 21 20:37:33.050081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG oslo_db.sqlalchemy.engines [None req-ccd00673-e618-4c85-9212-703d0069ca4e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87060) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 20:37:33.057625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writer thread running {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 20:37:33.243096 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG passlib.handlers.bcrypt [None req-ccd00673-e618-4c85-9212-703d0069ca4e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87060) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 20:37:33.243670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG passlib.handlers.bcrypt [None req-ccd00673-e618-4c85-9212-703d0069ca4e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87060) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 20:37:33.298754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-ccd00673-e618-4c85-9212-703d0069ca4e None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:33.300399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ccd00673-e618-4c85-9212-703d0069ca4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:33.426060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 2/3] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 20:37:33.503242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1a780-9c91-4718-bddb-2bb3bdebf82f None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:33.575493 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: INFO dbcounter [None req-7cf1a780-9c91-4718-bddb-2bb3bdebf82f None None] Registered counter for database keystone Jul 21 20:37:33.596542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG oslo_db.sqlalchemy.engines [None req-7cf1a780-9c91-4718-bddb-2bb3bdebf82f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87061) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 20:37:33.603926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writer thread running {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 20:37:33.733873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1a780-9c91-4718-bddb-2bb3bdebf82f None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:33.738375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df648d-44e0-4fcb-963b-35ac8e74a9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:33.739052 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df648d-44e0-4fcb-963b-35ac8e74a9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:33.739334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df648d-44e0-4fcb-963b-35ac8e74a9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:33.739611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df648d-44e0-4fcb-963b-35ac8e74a9e9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:33.749653 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df648d-44e0-4fcb-963b-35ac8e74a9e9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:33.802026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a7df648d-44e0-4fcb-963b-35ac8e74a9e9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 20:37:33.802292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df648d-44e0-4fcb-963b-35ac8e74a9e9 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:33.813890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 2/4] 10.222.0.60 () {58 vars in 1324 bytes} [Thu Jul 21 20:37:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 20:37:33.820221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd00673-e618-4c85-9212-703d0069ca4e None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:33.835293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd00673-e618-4c85-9212-703d0069ca4e None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:33.840290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5736c-f76e-49f5-b4d4-69f04d1e2979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:33.841436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5736c-f76e-49f5-b4d4-69f04d1e2979 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:33.841724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5736c-f76e-49f5-b4d4-69f04d1e2979 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:33.842001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5736c-f76e-49f5-b4d4-69f04d1e2979 admin admin] PATH_INFO: `/v3/projects/a9f8292e70024a6985791def05952106` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:33.846250 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5736c-f76e-49f5-b4d4-69f04d1e2979 admin admin] RBAC: Authorizing `identity:get_project(project_id=a9f8292e70024a6985791def05952106)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:33.897210 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e8a5736c-f76e-49f5-b4d4-69f04d1e2979 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 20:37:33.897543 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5736c-f76e-49f5-b4d4-69f04d1e2979 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:33.903195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 3/5] 10.222.0.60 () {56 vars in 1207 bytes} [Thu Jul 21 20:37:33 2022] GET /identity/v3/projects/a9f8292e70024a6985791def05952106 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:35.961289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d4ea1-89c3-4acc-b3a2-5e0d43e103b3 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:35.961833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d4ea1-89c3-4acc-b3a2-5e0d43e103b3 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:35.961944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d4ea1-89c3-4acc-b3a2-5e0d43e103b3 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:35.966617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 3/6] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:37:35.972276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e187c7c-d284-4702-a2a9-7b387ed769d9 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:35.972676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e187c7c-d284-4702-a2a9-7b387ed769d9 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:35.972839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e187c7c-d284-4702-a2a9-7b387ed769d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:35.996351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-9e187c7c-d284-4702-a2a9-7b387ed769d9 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:35.998373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-9e187c7c-d284-4702-a2a9-7b387ed769d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:36.009156 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 4/7] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 20:37:36.032324 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d4ea1-89c3-4acc-b3a2-5e0d43e103b3 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:36.048407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d4ea1-89c3-4acc-b3a2-5e0d43e103b3 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:36.052736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e0484-826c-4791-9cc6-dd76c1109461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:36.053391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e0484-826c-4791-9cc6-dd76c1109461 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:36.053679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e0484-826c-4791-9cc6-dd76c1109461 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:36.053959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e0484-826c-4791-9cc6-dd76c1109461 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:36.063044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e0484-826c-4791-9cc6-dd76c1109461 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:36.064873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e0484-826c-4791-9cc6-dd76c1109461 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:36.075008 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 4/8] 10.222.0.60 () {58 vars in 1324 bytes} [Thu Jul 21 20:37:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:37:36.083014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e187c7c-d284-4702-a2a9-7b387ed769d9 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:36.092694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e187c7c-d284-4702-a2a9-7b387ed769d9 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:36.097358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0d9c0-3ada-413a-913a-15da589363f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:36.098233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0d9c0-3ada-413a-913a-15da589363f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:36.098519 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0d9c0-3ada-413a-913a-15da589363f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:36.098797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0d9c0-3ada-413a-913a-15da589363f3 admin admin] PATH_INFO: `/v3/users/5ca745bbd8294b9eaae8eb567675a772` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:36.100260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0d9c0-3ada-413a-913a-15da589363f3 admin admin] RBAC: Authorizing `identity:get_user(user_id=5ca745bbd8294b9eaae8eb567675a772)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:36.102124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0d9c0-3ada-413a-913a-15da589363f3 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:36.103762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 5/9] 10.222.0.60 () {56 vars in 1198 bytes} [Thu Jul 21 20:37:36 2022] GET /identity/v3/users/5ca745bbd8294b9eaae8eb567675a772 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:38.221864 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee795cbf-1e49-4b14-8893-a13f813ef947 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:38.222390 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee795cbf-1e49-4b14-8893-a13f813ef947 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:38.222483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee795cbf-1e49-4b14-8893-a13f813ef947 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:38.223880 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 5/10] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:38.229039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834087e-4af1-44f9-89d1-8f69cedfb425 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:38.229712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834087e-4af1-44f9-89d1-8f69cedfb425 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:38.229860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834087e-4af1-44f9-89d1-8f69cedfb425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:38.246284 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-1834087e-4af1-44f9-89d1-8f69cedfb425 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:38.247478 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1834087e-4af1-44f9-89d1-8f69cedfb425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:38.291120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68909697-8106-4952-a573-856b3b4c4f4f None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:38.291400 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68909697-8106-4952-a573-856b3b4c4f4f None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:38.291677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68909697-8106-4952-a573-856b3b4c4f4f None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:38.294125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 6/11] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:38.299288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ebd24-70d5-49d0-a83c-84819aa23ffb None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:38.300054 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ebd24-70d5-49d0-a83c-84819aa23ffb None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:38.300342 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ebd24-70d5-49d0-a83c-84819aa23ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:38.331201 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 6/12] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:37:38.342103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG passlib.handlers.bcrypt [None req-716ebd24-70d5-49d0-a83c-84819aa23ffb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87061) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 20:37:38.342527 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG passlib.handlers.bcrypt [None req-716ebd24-70d5-49d0-a83c-84819aa23ffb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87061) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 20:37:38.354696 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834087e-4af1-44f9-89d1-8f69cedfb425 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:38.378270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:38.388932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-716ebd24-70d5-49d0-a83c-84819aa23ffb None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:38.390771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-716ebd24-70d5-49d0-a83c-84819aa23ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:38.473126 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834087e-4af1-44f9-89d1-8f69cedfb425 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:38.503925 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 7/13] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:37:38.512180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9402d96-255c-4685-8c40-56a8a18d2fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:38.513080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9402d96-255c-4685-8c40-56a8a18d2fd9 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:38.513442 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9402d96-255c-4685-8c40-56a8a18d2fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:38.513816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9402d96-255c-4685-8c40-56a8a18d2fd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:38.526967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ebd24-70d5-49d0-a83c-84819aa23ffb None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:38.528132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9402d96-255c-4685-8c40-56a8a18d2fd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-f9402d96-255c-4685-8c40-56a8a18d2fd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:37:38.542460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:37:38.543628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:37:38.544820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:37:38.544820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:37:38.544820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:37:38.544820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:37:38.544820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:37:38.544820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:37:38.544820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:37:38.544820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:37:38.544820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:37:38.544820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:37:38.544820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:37:38.544820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:37:38.544820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 7/14] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:37:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:37:38.549764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9402d96-255c-4685-8c40-56a8a18d2fd9 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:38.644172 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:38.651032 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ebd24-70d5-49d0-a83c-84819aa23ffb None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:38.655283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:38.666625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9402d96-255c-4685-8c40-56a8a18d2fd9 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:38.698042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514c4e7-9725-4ba7-84a9-60510179fdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:38.698958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514c4e7-9725-4ba7-84a9-60510179fdde None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:38.699244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514c4e7-9725-4ba7-84a9-60510179fdde None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:38.699525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514c4e7-9725-4ba7-84a9-60510179fdde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:38.705777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c4511-653e-4b86-a731-87d94369ee86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:38.706455 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c4511-653e-4b86-a731-87d94369ee86 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:38.706740 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c4511-653e-4b86-a731-87d94369ee86 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:38.707018 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c4511-653e-4b86-a731-87d94369ee86 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:38.707814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c4511-653e-4b86-a731-87d94369ee86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:38.708227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c4511-653e-4b86-a731-87d94369ee86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:38.709552 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c4511-653e-4b86-a731-87d94369ee86 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:38.722341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 8/15] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:37:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:37:38.728213 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c4511-653e-4b86-a731-87d94369ee86 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:38.778237 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514c4e7-9725-4ba7-84a9-60510179fdde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:38.779826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514c4e7-9725-4ba7-84a9-60510179fdde None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:38.833177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c4511-653e-4b86-a731-87d94369ee86 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:38.869422 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3211330-9b27-4a3d-84a6-a29e1d50028f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:38.870302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3211330-9b27-4a3d-84a6-a29e1d50028f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:38.870583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3211330-9b27-4a3d-84a6-a29e1d50028f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:38.870856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3211330-9b27-4a3d-84a6-a29e1d50028f None admin] PATH_INFO: `/v3/users/5ca745bbd8294b9eaae8eb567675a772` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:38.872130 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3211330-9b27-4a3d-84a6-a29e1d50028f None admin] RBAC: Authorizing `identity:get_user(user_id=5ca745bbd8294b9eaae8eb567675a772)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:38.873606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3211330-9b27-4a3d-84a6-a29e1d50028f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:38.875264 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 9/16] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:37:38 2022] GET /identity/v3/users/5ca745bbd8294b9eaae8eb567675a772 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:37:38.880717 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3211330-9b27-4a3d-84a6-a29e1d50028f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:38.885450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 8/17] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:37:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:37:38.890903 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514c4e7-9725-4ba7-84a9-60510179fdde None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:38.893765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:38.896737 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:38.986322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3211330-9b27-4a3d-84a6-a29e1d50028f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:39.004938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514c4e7-9725-4ba7-84a9-60510179fdde None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:39.022106 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84300647-aa89-4d64-a19b-fdd675c70c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:39.022902 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84300647-aa89-4d64-a19b-fdd675c70c10 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:39.023181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84300647-aa89-4d64-a19b-fdd675c70c10 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:39.023469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84300647-aa89-4d64-a19b-fdd675c70c10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:39.025247 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84300647-aa89-4d64-a19b-fdd675c70c10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:39.026705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84300647-aa89-4d64-a19b-fdd675c70c10 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:39.028676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 10/18] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:37:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:37:39.034311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84300647-aa89-4d64-a19b-fdd675c70c10 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:39.041697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9f5c6-e4e4-4701-94c2-448b9bf3d6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:39.042497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9f5c6-e4e4-4701-94c2-448b9bf3d6d5 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:39.042779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9f5c6-e4e4-4701-94c2-448b9bf3d6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:39.043052 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9f5c6-e4e4-4701-94c2-448b9bf3d6d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:39.063790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9f5c6-e4e4-4701-94c2-448b9bf3d6d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:39.065270 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9f5c6-e4e4-4701-94c2-448b9bf3d6d5 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:39.086990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-dab9f5c6-e4e4-4701-94c2-448b9bf3d6d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:37:39.086990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:37:39.086990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:37:39.086990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:37:39.086990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:37:39.086990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:37:39.086990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:37:39.086990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:37:39.086990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:37:39.086990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:37:39.086990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:37:39.086990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:37:39.086990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:37:39.086990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:37:39.086990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:37:39.087975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:37:39.088751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:37:39.088751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:37:39.088751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:37:39.088751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:37:39.088751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 9/19] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:37:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:37:39.094109 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9f5c6-e4e4-4701-94c2-448b9bf3d6d5 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:39.122339 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:39.141876 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84300647-aa89-4d64-a19b-fdd675c70c10 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:39.155627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:39.175879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01167d-3a12-405a-9c97-c93a542b0130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:39.176545 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01167d-3a12-405a-9c97-c93a542b0130 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:39.176833 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01167d-3a12-405a-9c97-c93a542b0130 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:39.177112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01167d-3a12-405a-9c97-c93a542b0130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:39.177955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01167d-3a12-405a-9c97-c93a542b0130 None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.domain.id=default, user.id=5ca745bbd8294b9eaae8eb567675a772) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:39.178267 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01167d-3a12-405a-9c97-c93a542b0130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:39.179733 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01167d-3a12-405a-9c97-c93a542b0130 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:39.187164 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 11/20] 10.222.0.60 () {56 vars in 1322 bytes} [Thu Jul 21 20:37:39 2022] GET /identity/v3/role_assignments?user.id=5ca745bbd8294b9eaae8eb567675a772&scope.domain.id=default&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:37:39.204034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9f5c6-e4e4-4701-94c2-448b9bf3d6d5 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:39.241953 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40653b-33e8-41b3-93a7-e333e320ed8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:39.242692 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40653b-33e8-41b3-93a7-e333e320ed8b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:39.242887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40653b-33e8-41b3-93a7-e333e320ed8b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:39.243160 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40653b-33e8-41b3-93a7-e333e320ed8b None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:39.244172 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40653b-33e8-41b3-93a7-e333e320ed8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:39.244477 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40653b-33e8-41b3-93a7-e333e320ed8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:39.245935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40653b-33e8-41b3-93a7-e333e320ed8b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:39.265983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 10/21] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:37:41.565870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c35186-0823-4a84-a898-bf77b08ca364 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:41.566975 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c35186-0823-4a84-a898-bf77b08ca364 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:41.567466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c35186-0823-4a84-a898-bf77b08ca364 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:41.569084 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 12/22] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:41.575133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f3cdd-014e-49a0-b1c6-6dfb56cf9590 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:41.575614 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f3cdd-014e-49a0-b1c6-6dfb56cf9590 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:41.575734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f3cdd-014e-49a0-b1c6-6dfb56cf9590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:41.601719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-712f3cdd-014e-49a0-b1c6-6dfb56cf9590 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:41.603087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-712f3cdd-014e-49a0-b1c6-6dfb56cf9590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:41.692263 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 11/23] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:37:41.738691 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c35186-0823-4a84-a898-bf77b08ca364 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:41.909532 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c35186-0823-4a84-a898-bf77b08ca364 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:41.937131 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbcba2-19d0-45fd-b00e-90e3303b911d None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:41.937726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbcba2-19d0-45fd-b00e-90e3303b911d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:41.938213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbcba2-19d0-45fd-b00e-90e3303b911d None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:41.947933 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 12/24] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:41 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:37:41.954289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0adcd-8bff-487d-9fd4-86b1bdb62361 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:41.954939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0adcd-8bff-487d-9fd4-86b1bdb62361 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:41.955260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a56eb-be73-4235-a47a-4c9b2c13a0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:41.955700 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0adcd-8bff-487d-9fd4-86b1bdb62361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:41.956086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a56eb-be73-4235-a47a-4c9b2c13a0f3 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:41.956373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a56eb-be73-4235-a47a-4c9b2c13a0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:41.956650 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a56eb-be73-4235-a47a-4c9b2c13a0f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:41.983739 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a56eb-be73-4235-a47a-4c9b2c13a0f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-082a56eb-be73-4235-a47a-4c9b2c13a0f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:37:41.986869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:37:41.987698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:37:41.988469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:37:41.988469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:37:41.988469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:37:41.988469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:37:41.988469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:37:41.988469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:37:41.988469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:37:41.988469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:37:41.988469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:37:41.988469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:37:41.988469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:37:41.988469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:37:41.992543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e9f0adcd-8bff-487d-9fd4-86b1bdb62361 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:41.993155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 13/25] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:37:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:37:41.994574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e9f0adcd-8bff-487d-9fd4-86b1bdb62361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:42.001294 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a56eb-be73-4235-a47a-4c9b2c13a0f3 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:42.016080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:42.082344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:42.148860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 13/26] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:37:42.150226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f75c1f-d3ba-499b-b3ba-dc382083cd95 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:42.150511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f75c1f-d3ba-499b-b3ba-dc382083cd95 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:42.150810 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f75c1f-d3ba-499b-b3ba-dc382083cd95 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:42.152309 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 14/27] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:42.158514 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba57749-f0c1-46ed-88e6-f23af5eacae4 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:42.158855 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba57749-f0c1-46ed-88e6-f23af5eacae4 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:42.159127 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba57749-f0c1-46ed-88e6-f23af5eacae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:42.168150 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a56eb-be73-4235-a47a-4c9b2c13a0f3 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:42.204986 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-9ba57749-f0c1-46ed-88e6-f23af5eacae4 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:42.206308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9ba57749-f0c1-46ed-88e6-f23af5eacae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:42.218291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45c52b-f50f-4c07-8a56-08e242c44341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:42.218973 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45c52b-f50f-4c07-8a56-08e242c44341 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:42.219256 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45c52b-f50f-4c07-8a56-08e242c44341 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:42.219529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45c52b-f50f-4c07-8a56-08e242c44341 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:42.220319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45c52b-f50f-4c07-8a56-08e242c44341 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:42.220623 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45c52b-f50f-4c07-8a56-08e242c44341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:42.222014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45c52b-f50f-4c07-8a56-08e242c44341 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:42.238972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 14/28] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:37:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:37:42.239627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45c52b-f50f-4c07-8a56-08e242c44341 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:42.287003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 15/29] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:37:42.288359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba57749-f0c1-46ed-88e6-f23af5eacae4 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:42.391863 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:42.463181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba57749-f0c1-46ed-88e6-f23af5eacae4 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:42.463348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45c52b-f50f-4c07-8a56-08e242c44341 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:42.472975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:42.509313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55dcb4-620a-4a40-8b54-91150ea83407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:42.510303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55dcb4-620a-4a40-8b54-91150ea83407 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:42.510606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8f852-7032-47e1-a881-52512feaa2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:42.510755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55dcb4-620a-4a40-8b54-91150ea83407 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:42.510980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55dcb4-620a-4a40-8b54-91150ea83407 None admin] PATH_INFO: `/v3/users/5ca745bbd8294b9eaae8eb567675a772` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:42.511419 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8f852-7032-47e1-a881-52512feaa2c8 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:42.511825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8f852-7032-47e1-a881-52512feaa2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:42.512115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8f852-7032-47e1-a881-52512feaa2c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:42.512782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55dcb4-620a-4a40-8b54-91150ea83407 None admin] RBAC: Authorizing `identity:get_user(user_id=5ca745bbd8294b9eaae8eb567675a772)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:42.514610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55dcb4-620a-4a40-8b54-91150ea83407 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:42.516687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 16/30] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:37:42 2022] GET /identity/v3/users/5ca745bbd8294b9eaae8eb567675a772 => generated 247 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:42.517427 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55dcb4-620a-4a40-8b54-91150ea83407 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:42.526947 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8f852-7032-47e1-a881-52512feaa2c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:42.528461 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8f852-7032-47e1-a881-52512feaa2c8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:42.547774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-c3b8f852-7032-47e1-a881-52512feaa2c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:37:42.547774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:37:42.547774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:37:42.547774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:37:42.547774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:37:42.547774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:37:42.547774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:37:42.547774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:37:42.547774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:37:42.547774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:37:42.547774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:37:42.547774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:37:42.547774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:37:42.547774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:37:42.547774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:37:42.548607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:37:42.549378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:37:42.549378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:37:42.549378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:37:42.549378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:37:42.549378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 15/31] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:37:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:37:42.555857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbac9ec-16f3-42cf-9f19-c40b6a62a642 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:42.555857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbac9ec-16f3-42cf-9f19-c40b6a62a642 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:42.555857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbac9ec-16f3-42cf-9f19-c40b6a62a642 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:42.555857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 16/32] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:42.555857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbac9ec-16f3-42cf-9f19-c40b6a62a642 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:42.647350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55dcb4-620a-4a40-8b54-91150ea83407 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:42.677894 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbac9ec-16f3-42cf-9f19-c40b6a62a642 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:42.689772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4364ea-7290-4136-a3b3-9af4e4286858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:42.690474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4364ea-7290-4136-a3b3-9af4e4286858 None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:42.690762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4364ea-7290-4136-a3b3-9af4e4286858 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:42.691040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4364ea-7290-4136-a3b3-9af4e4286858 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:42.691906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4364ea-7290-4136-a3b3-9af4e4286858 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:42.693447 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4364ea-7290-4136-a3b3-9af4e4286858 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:42.708465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:42.729147 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc63891-b92f-4082-b340-bd4f83d1f851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:42.730300 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc63891-b92f-4082-b340-bd4f83d1f851 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:42.730716 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc63891-b92f-4082-b340-bd4f83d1f851 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:42.731092 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc63891-b92f-4082-b340-bd4f83d1f851 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:42.732459 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc63891-b92f-4082-b340-bd4f83d1f851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:42.734152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc63891-b92f-4082-b340-bd4f83d1f851 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:42.736662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 17/33] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:37:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:42.737609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc63891-b92f-4082-b340-bd4f83d1f851 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:42.855017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 17/34] 10.222.0.60 () {60 vars in 1131 bytes} [Thu Jul 21 20:37:42 2022] POST /identity/v3/roles => generated 219 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:37:42.856741 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268b09a-967d-4978-9ee8-c22fae3af310 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:42.857134 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268b09a-967d-4978-9ee8-c22fae3af310 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:42.857535 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268b09a-967d-4978-9ee8-c22fae3af310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:42.896563 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-f268b09a-967d-4978-9ee8-c22fae3af310 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:42.898177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f268b09a-967d-4978-9ee8-c22fae3af310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:42.945106 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc63891-b92f-4082-b340-bd4f83d1f851 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:42.964334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:43.021864 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac83232-614a-4484-934e-e6d6c944573b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:43.022459 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac83232-614a-4484-934e-e6d6c944573b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:43.022739 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac83232-614a-4484-934e-e6d6c944573b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:43.023011 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac83232-614a-4484-934e-e6d6c944573b None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:43.024314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac83232-614a-4484-934e-e6d6c944573b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:43.024626 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac83232-614a-4484-934e-e6d6c944573b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:43.026048 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac83232-614a-4484-934e-e6d6c944573b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:43.050128 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 18/35] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:43.051096 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac83232-614a-4484-934e-e6d6c944573b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:43.067377 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 18/36] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:37:43.068061 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268b09a-967d-4978-9ee8-c22fae3af310 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:43.133669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:43.219326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268b09a-967d-4978-9ee8-c22fae3af310 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:43.235232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac83232-614a-4484-934e-e6d6c944573b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:43.261998 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3283f51-fcf0-4ab9-a2a9-81015eaf7d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:43.262660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3283f51-fcf0-4ab9-a2a9-81015eaf7d86 None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:43.262951 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3283f51-fcf0-4ab9-a2a9-81015eaf7d86 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:43.263231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3283f51-fcf0-4ab9-a2a9-81015eaf7d86 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:43.263990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3283f51-fcf0-4ab9-a2a9-81015eaf7d86 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:43.265570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3283f51-fcf0-4ab9-a2a9-81015eaf7d86 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:43.273197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a7833-595a-43cf-84af-724f4bc3f592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:43.274692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a7833-595a-43cf-84af-724f4bc3f592 None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:43.275430 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a7833-595a-43cf-84af-724f4bc3f592 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:43.275744 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a7833-595a-43cf-84af-724f4bc3f592 None admin] PATH_INFO: `/v3/domains/default/users/5ca745bbd8294b9eaae8eb567675a772/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:43.278243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a7833-595a-43cf-84af-724f4bc3f592 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5ca745bbd8294b9eaae8eb567675a772, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:43.279722 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a7833-595a-43cf-84af-724f4bc3f592 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:43.294949 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 19/37] 10.222.0.60 () {60 vars in 1141 bytes} [Thu Jul 21 20:37:43 2022] POST /identity/v3/projects => generated 301 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:37:43.296804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3283f51-fcf0-4ab9-a2a9-81015eaf7d86 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:43.323369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:43.425739 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 19/38] 10.222.0.60 () {58 vars in 1361 bytes} [Thu Jul 21 20:37:43 2022] PUT /identity/v3/domains/default/users/5ca745bbd8294b9eaae8eb567675a772/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:37:43.428342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac869f-2eb1-4544-aac6-ea4a50755748 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:43.428778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac869f-2eb1-4544-aac6-ea4a50755748 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:43.429178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac869f-2eb1-4544-aac6-ea4a50755748 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:43.435759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 20/39] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:43.441704 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c6784-f611-4280-bb80-0299770502b8 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:43.442017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c6784-f611-4280-bb80-0299770502b8 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:43.442264 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c6784-f611-4280-bb80-0299770502b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:43.443193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3283f51-fcf0-4ab9-a2a9-81015eaf7d86 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:43.472250 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-da8c6784-f611-4280-bb80-0299770502b8 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:43.474475 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-da8c6784-f611-4280-bb80-0299770502b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:43.522849 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7183be-8543-4add-96b7-b0c34f622038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:43.523676 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7183be-8543-4add-96b7-b0c34f622038 None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:43.524586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7183be-8543-4add-96b7-b0c34f622038 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:43.524963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7183be-8543-4add-96b7-b0c34f622038 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:43.526140 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7183be-8543-4add-96b7-b0c34f622038 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:43.528048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7183be-8543-4add-96b7-b0c34f622038 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:43.551675 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 20/40] 10.222.0.60 () {60 vars in 1131 bytes} [Thu Jul 21 20:37:43 2022] POST /identity/v3/roles => generated 225 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:37:43.553199 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3a39c-87d5-4bee-a98d-a6913dd94e0d None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:43.553499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3a39c-87d5-4bee-a98d-a6913dd94e0d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:43.553802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3a39c-87d5-4bee-a98d-a6913dd94e0d None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:43.556178 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 21/41] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:43.563773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0bd35-0260-42fa-8a84-143cebf2adf8 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:43.563773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0bd35-0260-42fa-8a84-143cebf2adf8 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:43.564023 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0bd35-0260-42fa-8a84-143cebf2adf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:43.597999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e1c0bd35-0260-42fa-8a84-143cebf2adf8 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:43.600107 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e1c0bd35-0260-42fa-8a84-143cebf2adf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:43.601520 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 21/42] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:37:43.627416 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c6784-f611-4280-bb80-0299770502b8 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:43.671824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:43.700790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 22/43] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:37:43.725940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0bd35-0260-42fa-8a84-143cebf2adf8 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:43.735776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:43.858031 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0bd35-0260-42fa-8a84-143cebf2adf8 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:43.872561 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c6784-f611-4280-bb80-0299770502b8 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:43.898531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64915c-7e06-4a48-8724-4aef41a66f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:43.899753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64915c-7e06-4a48-8724-4aef41a66f8f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:43.900038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64915c-7e06-4a48-8724-4aef41a66f8f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:43.900320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64915c-7e06-4a48-8724-4aef41a66f8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:43.901594 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba64915c-7e06-4a48-8724-4aef41a66f8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:43.903072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba64915c-7e06-4a48-8724-4aef41a66f8f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:43.904946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 23/44] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:37:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:37:43.910388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64915c-7e06-4a48-8724-4aef41a66f8f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:43.911242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567f326-8959-4ab5-9edc-8ea8d29ae541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:43.912164 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567f326-8959-4ab5-9edc-8ea8d29ae541 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:43.912625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567f326-8959-4ab5-9edc-8ea8d29ae541 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:43.913082 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567f326-8959-4ab5-9edc-8ea8d29ae541 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:43.914148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567f326-8959-4ab5-9edc-8ea8d29ae541 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:43.916992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567f326-8959-4ab5-9edc-8ea8d29ae541 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:43.937065 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 22/45] 10.222.0.60 () {60 vars in 1131 bytes} [Thu Jul 21 20:37:43 2022] POST /identity/v3/roles => generated 223 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:37:44.008936 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:44.037711 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64915c-7e06-4a48-8724-4aef41a66f8f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:44.078632 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0c53d-5fe1-489f-9559-3cc83610bfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:44.079659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0c53d-5fe1-489f-9559-3cc83610bfc7 None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:44.080114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0c53d-5fe1-489f-9559-3cc83610bfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:44.080530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0c53d-5fe1-489f-9559-3cc83610bfc7 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:44.081420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0c53d-5fe1-489f-9559-3cc83610bfc7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:44.083047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0c53d-5fe1-489f-9559-3cc83610bfc7 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:44.120587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 24/46] 10.222.0.60 () {60 vars in 1141 bytes} [Thu Jul 21 20:37:43 2022] POST /identity/v3/projects => generated 312 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:37:45.589917 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0175db9-e4b9-42da-a0f8-3216c499fd8b None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:45.590426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0175db9-e4b9-42da-a0f8-3216c499fd8b None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:45.590528 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0175db9-e4b9-42da-a0f8-3216c499fd8b None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:45.591908 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 23/47] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:45.597214 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea694592-e5fa-477c-8d70-4f9c0168e31e None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:45.597638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea694592-e5fa-477c-8d70-4f9c0168e31e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:45.597793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea694592-e5fa-477c-8d70-4f9c0168e31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:45.615004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-ea694592-e5fa-477c-8d70-4f9c0168e31e None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:45.616256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ea694592-e5fa-477c-8d70-4f9c0168e31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:45.685438 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 25/48] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:37:45.708633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0175db9-e4b9-42da-a0f8-3216c499fd8b None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:45.806587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:45.815097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0175db9-e4b9-42da-a0f8-3216c499fd8b None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:45.849810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4bcc7-fe8f-4956-bc58-7d6e7dda9abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:45.850607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4bcc7-fe8f-4956-bc58-7d6e7dda9abe None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:45.850894 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4bcc7-fe8f-4956-bc58-7d6e7dda9abe None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:45.851170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4bcc7-fe8f-4956-bc58-7d6e7dda9abe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:45.861882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4bcc7-fe8f-4956-bc58-7d6e7dda9abe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-f7e4bcc7-fe8f-4956-bc58-7d6e7dda9abe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:37:45.870208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:37:45.871249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:37:45.872682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:37:45.872682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:37:45.872682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:37:45.872682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:37:45.872682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:37:45.872682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:37:45.872682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:37:45.872682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:37:45.872682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:37:45.872682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:37:45.872682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:37:45.872682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:37:45.872682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 24/49] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:37:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:37:45.877968 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea694592-e5fa-477c-8d70-4f9c0168e31e None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:45.936794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:45.982631 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea694592-e5fa-477c-8d70-4f9c0168e31e None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:46.017054 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23799fd0-6e48-4ca7-9057-55123b7225a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:46.017729 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23799fd0-6e48-4ca7-9057-55123b7225a0 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:46.018014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23799fd0-6e48-4ca7-9057-55123b7225a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:46.018290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23799fd0-6e48-4ca7-9057-55123b7225a0 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:46.019052 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23799fd0-6e48-4ca7-9057-55123b7225a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:46.019364 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23799fd0-6e48-4ca7-9057-55123b7225a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:46.020805 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23799fd0-6e48-4ca7-9057-55123b7225a0 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:46.034261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 26/50] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:37:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:46.041201 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4bcc7-fe8f-4956-bc58-7d6e7dda9abe None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:46.154115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4bcc7-fe8f-4956-bc58-7d6e7dda9abe None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:46.190588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1badbf-0c38-493f-bdaa-69e21ed969e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:46.191388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1badbf-0c38-493f-bdaa-69e21ed969e5 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:46.191675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1badbf-0c38-493f-bdaa-69e21ed969e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:46.191992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1badbf-0c38-493f-bdaa-69e21ed969e5 None admin] PATH_INFO: `/v3/users/5ca745bbd8294b9eaae8eb567675a772` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:46.193207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1badbf-0c38-493f-bdaa-69e21ed969e5 None admin] RBAC: Authorizing `identity:get_user(user_id=5ca745bbd8294b9eaae8eb567675a772)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:46.194675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1badbf-0c38-493f-bdaa-69e21ed969e5 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:46.197047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 25/51] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:37:46 2022] GET /identity/v3/users/5ca745bbd8294b9eaae8eb567675a772 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:46.202440 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23799fd0-6e48-4ca7-9057-55123b7225a0 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:46.212285 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769c782-1a17-4139-b8d9-390a57b56d79 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:46.212599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769c782-1a17-4139-b8d9-390a57b56d79 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:46.212925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769c782-1a17-4139-b8d9-390a57b56d79 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:46.214377 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 26/52] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:37:46.219321 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be107403-e67a-4720-8d17-29aac4cf5a56 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:46.219619 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be107403-e67a-4720-8d17-29aac4cf5a56 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:46.219941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be107403-e67a-4720-8d17-29aac4cf5a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:46.237309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-be107403-e67a-4720-8d17-29aac4cf5a56 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:46.238529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-be107403-e67a-4720-8d17-29aac4cf5a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:46.257165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:46.309177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23799fd0-6e48-4ca7-9057-55123b7225a0 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:46.310959 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 27/53] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:37:46.335195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be107403-e67a-4720-8d17-29aac4cf5a56 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:46.433542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:46.446213 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be107403-e67a-4720-8d17-29aac4cf5a56 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:46.451451 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325c641-8cec-4d74-a24c-9b3422b1a89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:46.452282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325c641-8cec-4d74-a24c-9b3422b1a89e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:46.452571 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325c641-8cec-4d74-a24c-9b3422b1a89e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:46.452851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325c641-8cec-4d74-a24c-9b3422b1a89e None admin] PATH_INFO: `/v3/domains/default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:46.454051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325c641-8cec-4d74-a24c-9b3422b1a89e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:46.455542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325c641-8cec-4d74-a24c-9b3422b1a89e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:46.457226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 27/54] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:37:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:37:46.463170 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325c641-8cec-4d74-a24c-9b3422b1a89e None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:46.482384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b49d7c-894c-4e69-be40-fd099a6cc842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:46.483190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b49d7c-894c-4e69-be40-fd099a6cc842 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:46.483480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b49d7c-894c-4e69-be40-fd099a6cc842 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:46.483775 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b49d7c-894c-4e69-be40-fd099a6cc842 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:46.484982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b49d7c-894c-4e69-be40-fd099a6cc842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:46.486455 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b49d7c-894c-4e69-be40-fd099a6cc842 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:46.488159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 28/55] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:37:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:37:46.494237 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b49d7c-894c-4e69-be40-fd099a6cc842 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:46.524597 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:46.567014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325c641-8cec-4d74-a24c-9b3422b1a89e None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:46.601520 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b49d7c-894c-4e69-be40-fd099a6cc842 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:46.603241 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00091a-6b87-4c2c-bdf6-92d1f1507f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:46.604097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00091a-6b87-4c2c-bdf6-92d1f1507f97 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:46.604642 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00091a-6b87-4c2c-bdf6-92d1f1507f97 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:46.604918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00091a-6b87-4c2c-bdf6-92d1f1507f97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:46.605756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00091a-6b87-4c2c-bdf6-92d1f1507f97 None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.domain.id=default, user.id=5ca745bbd8294b9eaae8eb567675a772) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:46.606062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00091a-6b87-4c2c-bdf6-92d1f1507f97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:46.607502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00091a-6b87-4c2c-bdf6-92d1f1507f97 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:46.616573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 28/56] 10.222.0.60 () {56 vars in 1322 bytes} [Thu Jul 21 20:37:46 2022] GET /identity/v3/role_assignments?user.id=5ca745bbd8294b9eaae8eb567675a772&scope.domain.id=default&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:37:46.642648 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0cc43-f06c-439c-a27e-e374ef7e358c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:46.643338 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0cc43-f06c-439c-a27e-e374ef7e358c None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:46.643753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0cc43-f06c-439c-a27e-e374ef7e358c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:46.643930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0cc43-f06c-439c-a27e-e374ef7e358c None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:46.644689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0cc43-f06c-439c-a27e-e374ef7e358c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:46.646223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0cc43-f06c-439c-a27e-e374ef7e358c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:46.677278 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 29/57] 10.222.0.60 () {60 vars in 1140 bytes} [Thu Jul 21 20:37:46 2022] POST /identity/v3/projects => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:37:48.743064 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d842f-8e57-47f3-af7b-7f40a740426b None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:48.743566 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d842f-8e57-47f3-af7b-7f40a740426b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:48.743566 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d842f-8e57-47f3-af7b-7f40a740426b None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:48.744977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 29/58] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:48.750391 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb192b-bcba-419a-b479-90ba2bf7fbde None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:48.750801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb192b-bcba-419a-b479-90ba2bf7fbde None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:48.750962 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb192b-bcba-419a-b479-90ba2bf7fbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:48.768574 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-3ddb192b-bcba-419a-b479-90ba2bf7fbde None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:48.769797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-3ddb192b-bcba-419a-b479-90ba2bf7fbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:48.844430 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 30/59] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:37:48.869535 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d842f-8e57-47f3-af7b-7f40a740426b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:48.924861 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:48.986868 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d842f-8e57-47f3-af7b-7f40a740426b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:49.023101 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dad0ed-c9a3-479d-80cf-f235bfced357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:49.024459 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dad0ed-c9a3-479d-80cf-f235bfced357 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:49.024746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dad0ed-c9a3-479d-80cf-f235bfced357 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:49.025028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dad0ed-c9a3-479d-80cf-f235bfced357 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:49.026238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dad0ed-c9a3-479d-80cf-f235bfced357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:49.027691 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dad0ed-c9a3-479d-80cf-f235bfced357 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:49.029363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 30/60] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:37:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:49.035019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb192b-bcba-419a-b479-90ba2bf7fbde None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:49.038334 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:49.155810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb192b-bcba-419a-b479-90ba2bf7fbde None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:49.190106 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40e51c-8c1d-4355-9ac5-2cb8384c6fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:49.190783 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40e51c-8c1d-4355-9ac5-2cb8384c6fa0 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:49.191072 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40e51c-8c1d-4355-9ac5-2cb8384c6fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:49.191350 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40e51c-8c1d-4355-9ac5-2cb8384c6fa0 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:49.192109 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40e51c-8c1d-4355-9ac5-2cb8384c6fa0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:49.193587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40e51c-8c1d-4355-9ac5-2cb8384c6fa0 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:49.220361 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 31/61] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:37:49 2022] POST /identity/v3/users => generated 275 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:37:52.388086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bd179-e387-4d12-87b2-3274ddef09ff None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:52.388589 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bd179-e387-4d12-87b2-3274ddef09ff None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:52.388661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bd179-e387-4d12-87b2-3274ddef09ff None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:52.390436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 31/62] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:52.395369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef9a5d-7166-4b14-8e25-e496ffe79945 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:52.396161 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef9a5d-7166-4b14-8e25-e496ffe79945 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:52.396623 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef9a5d-7166-4b14-8e25-e496ffe79945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:52.420778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-64ef9a5d-7166-4b14-8e25-e496ffe79945 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:52.422018 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-64ef9a5d-7166-4b14-8e25-e496ffe79945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:52.509369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:52.526655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 32/63] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:37:52.561482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bd179-e387-4d12-87b2-3274ddef09ff None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:52.625436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dfae2-dd02-47d4-9b50-f1c7cdfa924a None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:52.626066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dfae2-dd02-47d4-9b50-f1c7cdfa924a None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:52.626901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dfae2-dd02-47d4-9b50-f1c7cdfa924a None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:52.628839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 33/64] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:52.635960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7545d-6863-4a60-9f40-1913d5b3148f None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:52.636509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7545d-6863-4a60-9f40-1913d5b3148f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:52.636988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7545d-6863-4a60-9f40-1913d5b3148f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:52.665533 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-f9d7545d-6863-4a60-9f40-1913d5b3148f None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:52.668992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f9d7545d-6863-4a60-9f40-1913d5b3148f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:52.748407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bd179-e387-4d12-87b2-3274ddef09ff None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:52.796107 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:52.817595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc881eb-341d-4a73-b2be-8a28fc8832b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:52.818428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc881eb-341d-4a73-b2be-8a28fc8832b0 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:52.818713 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc881eb-341d-4a73-b2be-8a28fc8832b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:52.818996 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc881eb-341d-4a73-b2be-8a28fc8832b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:52.824136 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 34/65] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:37:52.837129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc881eb-341d-4a73-b2be-8a28fc8832b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:37:52.838634 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe5161-5554-4565-a451-b4ab17a777cc None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:52.838927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe5161-5554-4565-a451-b4ab17a777cc None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:52.839210 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe5161-5554-4565-a451-b4ab17a777cc None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:52.840622 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 35/66] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:52.845945 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da842d-6d1a-4075-947e-24b6869071dc None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:52.846236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da842d-6d1a-4075-947e-24b6869071dc None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:52.846522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da842d-6d1a-4075-947e-24b6869071dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-afc881eb-341d-4a73-b2be-8a28fc8832b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:37:52.857141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:37:52.858018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:37:52.858755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:37:52.858755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:37:52.858755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:37:52.858755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:37:52.858755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:37:52.858755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:37:52.858755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:37:52.858755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:37:52.858755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:37:52.858755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:37:52.858755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:37:52.858755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:37:52.858755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 32/67] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:37:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:37:52.866423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc881eb-341d-4a73-b2be-8a28fc8832b0 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:52.874518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c1da842d-6d1a-4075-947e-24b6869071dc None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:52.876008 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c1da842d-6d1a-4075-947e-24b6869071dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:52.972373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 36/68] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:37:52.972971 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da842d-6d1a-4075-947e-24b6869071dc None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:53.002346 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc881eb-341d-4a73-b2be-8a28fc8832b0 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:53.045084 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:53.047028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f89bd-ad5c-4174-a645-8ebb3bf3fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:53.047708 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f89bd-ad5c-4174-a645-8ebb3bf3fcea None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:53.048021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f89bd-ad5c-4174-a645-8ebb3bf3fcea None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:53.048295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f89bd-ad5c-4174-a645-8ebb3bf3fcea None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:53.049857 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f89bd-ad5c-4174-a645-8ebb3bf3fcea None admin] RBAC: Adding query filter params (name=member) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:53.050017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f89bd-ad5c-4174-a645-8ebb3bf3fcea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:53.051641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f89bd-ad5c-4174-a645-8ebb3bf3fcea None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:53.067891 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 33/69] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:37:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:37:53.069023 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72658906-9df1-4fa2-b1ba-faa48585a819 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:53.069320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72658906-9df1-4fa2-b1ba-faa48585a819 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:53.069591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72658906-9df1-4fa2-b1ba-faa48585a819 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:53.075956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 34/70] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:53.075956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72658906-9df1-4fa2-b1ba-faa48585a819 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:53.126992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da842d-6d1a-4075-947e-24b6869071dc None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:53.142353 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:53.175621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eaaa4c-e426-4877-9db8-8ed14033cc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:53.176471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eaaa4c-e426-4877-9db8-8ed14033cc6a None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:53.176761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eaaa4c-e426-4877-9db8-8ed14033cc6a None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:53.177043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eaaa4c-e426-4877-9db8-8ed14033cc6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:53.189446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eaaa4c-e426-4877-9db8-8ed14033cc6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-42eaaa4c-e426-4877-9db8-8ed14033cc6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:37:53.197187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:37:53.198054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:37:53.198769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:37:53.198769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:37:53.198769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:37:53.198769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:37:53.198769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:37:53.198769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:37:53.198769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:37:53.198769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:37:53.198769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:37:53.198769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:37:53.198769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:37:53.198769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:37:53.198769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 37/71] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:37:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:37:53.199439 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eaaa4c-e426-4877-9db8-8ed14033cc6a None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:53.224112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72658906-9df1-4fa2-b1ba-faa48585a819 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:53.264510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91732a09-bd5c-47c1-8e8f-8cba28b315f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:53.265434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91732a09-bd5c-47c1-8e8f-8cba28b315f7 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:53.265802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91732a09-bd5c-47c1-8e8f-8cba28b315f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:53.266153 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91732a09-bd5c-47c1-8e8f-8cba28b315f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:53.277499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91732a09-bd5c-47c1-8e8f-8cba28b315f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-91732a09-bd5c-47c1-8e8f-8cba28b315f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:37:53.285669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:37:53.286543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:37:53.287282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:37:53.287282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:37:53.287282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:37:53.287282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:37:53.287282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:37:53.287282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:37:53.287282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:37:53.287282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:37:53.287282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:37:53.287282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:37:53.287282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:37:53.287282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:37:53.287282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 35/72] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:37:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:37:53.288236 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153a730-458d-4e2c-b446-2b1669d9ce1f None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:53.288428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153a730-458d-4e2c-b446-2b1669d9ce1f None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:53.288717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153a730-458d-4e2c-b446-2b1669d9ce1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:53.307173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-8153a730-458d-4e2c-b446-2b1669d9ce1f None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:53.308443 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8153a730-458d-4e2c-b446-2b1669d9ce1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:53.317093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eaaa4c-e426-4877-9db8-8ed14033cc6a None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:53.327645 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:53.353382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9305f4e-c7a7-4539-bad6-23e6d6a8f436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:53.354526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9305f4e-c7a7-4539-bad6-23e6d6a8f436 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:53.354816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9305f4e-c7a7-4539-bad6-23e6d6a8f436 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:53.355095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9305f4e-c7a7-4539-bad6-23e6d6a8f436 None admin] PATH_INFO: `/v3/users/805b77c8bb67450db935957c46dc8b9b` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:53.369241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9305f4e-c7a7-4539-bad6-23e6d6a8f436 None admin] RBAC: Authorizing `identity:get_user(user_id=805b77c8bb67450db935957c46dc8b9b)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:53.370729 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9305f4e-c7a7-4539-bad6-23e6d6a8f436 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:53.372516 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 38/73] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:37:53 2022] GET /identity/v3/users/805b77c8bb67450db935957c46dc8b9b => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:53.373298 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9305f4e-c7a7-4539-bad6-23e6d6a8f436 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:53.378061 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 36/74] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:37:53.379409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6858d-e81a-4593-b3d3-f43bd8e20504 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:53.379701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6858d-e81a-4593-b3d3-f43bd8e20504 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:53.380431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6858d-e81a-4593-b3d3-f43bd8e20504 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:53.381578 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 37/75] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:53.382286 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6858d-e81a-4593-b3d3-f43bd8e20504 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:53.420601 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:53.490600 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9305f4e-c7a7-4539-bad6-23e6d6a8f436 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:53.493929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6858d-e81a-4593-b3d3-f43bd8e20504 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:53.526101 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48268447-1fa0-432c-85f4-accd8abd089b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:53.526787 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48268447-1fa0-432c-85f4-accd8abd089b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:53.527075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48268447-1fa0-432c-85f4-accd8abd089b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:53.527363 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48268447-1fa0-432c-85f4-accd8abd089b None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:53.527867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1244b0-6313-43dc-9a6c-2da0bfe17c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:53.528164 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48268447-1fa0-432c-85f4-accd8abd089b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:53.528477 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48268447-1fa0-432c-85f4-accd8abd089b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:53.528561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1244b0-6313-43dc-9a6c-2da0bfe17c6c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:53.528804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1244b0-6313-43dc-9a6c-2da0bfe17c6c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:53.529081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1244b0-6313-43dc-9a6c-2da0bfe17c6c None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:53.529893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48268447-1fa0-432c-85f4-accd8abd089b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:53.530092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1244b0-6313-43dc-9a6c-2da0bfe17c6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:53.530401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1244b0-6313-43dc-9a6c-2da0bfe17c6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:53.531809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1244b0-6313-43dc-9a6c-2da0bfe17c6c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:53.538391 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 39/76] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:37:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:53.539938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 38/77] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:37:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:53.541434 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48268447-1fa0-432c-85f4-accd8abd089b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:53.542318 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb979d-bac1-4c5e-a0cd-d96a58f00082 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:53.542795 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb979d-bac1-4c5e-a0cd-d96a58f00082 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:53.543248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb979d-bac1-4c5e-a0cd-d96a58f00082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:53.562283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-dedb979d-bac1-4c5e-a0cd-d96a58f00082 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:53.563568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-dedb979d-bac1-4c5e-a0cd-d96a58f00082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:53.579477 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:53.632204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 39/78] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:37:53.633190 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb979d-bac1-4c5e-a0cd-d96a58f00082 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:53.659036 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48268447-1fa0-432c-85f4-accd8abd089b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:53.662962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:53.696247 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f2624-cfe1-415a-a8af-202cb00d504c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:53.697106 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f2624-cfe1-415a-a8af-202cb00d504c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:53.697397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f2624-cfe1-415a-a8af-202cb00d504c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:53.697683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f2624-cfe1-415a-a8af-202cb00d504c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:53.742458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb979d-bac1-4c5e-a0cd-d96a58f00082 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:53.767298 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f2624-cfe1-415a-a8af-202cb00d504c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:53.768842 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f2624-cfe1-415a-a8af-202cb00d504c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:53.777891 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02b184-0605-4b19-8557-a1fd51460412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:53.778696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02b184-0605-4b19-8557-a1fd51460412 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:53.778982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02b184-0605-4b19-8557-a1fd51460412 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:53.779259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02b184-0605-4b19-8557-a1fd51460412 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:53.790187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02b184-0605-4b19-8557-a1fd51460412 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-5d02b184-0605-4b19-8557-a1fd51460412 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:37:53.798057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:37:53.799082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:37:53.799907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:37:53.799907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:37:53.799907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:37:53.799907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:37:53.799907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:37:53.799907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:37:53.799907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:37:53.799907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:37:53.799907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:37:53.799907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:37:53.799907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:37:53.799907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:37:53.799907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 40/79] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:37:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:37:53.800510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02b184-0605-4b19-8557-a1fd51460412 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:53.805535 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:53.885881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:53.886081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 40/80] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:37:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:37:53.886935 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f2624-cfe1-415a-a8af-202cb00d504c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:53.908268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02b184-0605-4b19-8557-a1fd51460412 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:53.944001 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53578f47-b0c4-4903-bff7-20ef89faf3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:53.944890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53578f47-b0c4-4903-bff7-20ef89faf3a5 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:53.945680 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53578f47-b0c4-4903-bff7-20ef89faf3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:53.945965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53578f47-b0c4-4903-bff7-20ef89faf3a5 None admin] PATH_INFO: `/v3/users/5ca745bbd8294b9eaae8eb567675a772` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:53.947233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53578f47-b0c4-4903-bff7-20ef89faf3a5 None admin] RBAC: Authorizing `identity:get_user(user_id=5ca745bbd8294b9eaae8eb567675a772)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:53.948745 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53578f47-b0c4-4903-bff7-20ef89faf3a5 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:53.950410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 41/81] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:37:53 2022] GET /identity/v3/users/5ca745bbd8294b9eaae8eb567675a772 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:53.951189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53578f47-b0c4-4903-bff7-20ef89faf3a5 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:53.994578 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f2624-cfe1-415a-a8af-202cb00d504c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:54.024328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:54.030907 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad64be-c970-4db7-b203-8a3e59b5d1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:54.031905 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad64be-c970-4db7-b203-8a3e59b5d1e9 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:54.032198 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad64be-c970-4db7-b203-8a3e59b5d1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:54.032479 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad64be-c970-4db7-b203-8a3e59b5d1e9 None admin] PATH_INFO: `/v3/users/805b77c8bb67450db935957c46dc8b9b` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:54.033736 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad64be-c970-4db7-b203-8a3e59b5d1e9 None admin] RBAC: Authorizing `identity:get_user(user_id=805b77c8bb67450db935957c46dc8b9b)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:54.035196 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad64be-c970-4db7-b203-8a3e59b5d1e9 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:54.036955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 41/82] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:37:53 2022] GET /identity/v3/users/805b77c8bb67450db935957c46dc8b9b => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:54.037725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad64be-c970-4db7-b203-8a3e59b5d1e9 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:54.063646 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53578f47-b0c4-4903-bff7-20ef89faf3a5 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:54.097961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec20c5-d19b-49f8-8bbd-2b10f707241f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:54.098757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec20c5-d19b-49f8-8bbd-2b10f707241f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:54.099048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec20c5-d19b-49f8-8bbd-2b10f707241f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:54.099330 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec20c5-d19b-49f8-8bbd-2b10f707241f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:54.100560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec20c5-d19b-49f8-8bbd-2b10f707241f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:54.102663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec20c5-d19b-49f8-8bbd-2b10f707241f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:54.104599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 42/83] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:37:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:54.105378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec20c5-d19b-49f8-8bbd-2b10f707241f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:54.125857 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:54.147017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad64be-c970-4db7-b203-8a3e59b5d1e9 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:54.188437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1321c-be25-4803-94d2-a8b855efe514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:54.189137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1321c-be25-4803-94d2-a8b855efe514 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:54.189431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1321c-be25-4803-94d2-a8b855efe514 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:54.189711 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1321c-be25-4803-94d2-a8b855efe514 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:54.190477 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1321c-be25-4803-94d2-a8b855efe514 None admin] RBAC: Adding query filter params (name=member) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:54.190789 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1321c-be25-4803-94d2-a8b855efe514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:54.192316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1321c-be25-4803-94d2-a8b855efe514 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:54.202283 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 42/84] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:37:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:54.203134 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1321c-be25-4803-94d2-a8b855efe514 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:54.213556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec20c5-d19b-49f8-8bbd-2b10f707241f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:54.248636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61846bd0-6e78-42c8-8648-6986c0b9de2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:54.249421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61846bd0-6e78-42c8-8648-6986c0b9de2f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:54.249707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61846bd0-6e78-42c8-8648-6986c0b9de2f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:54.249988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61846bd0-6e78-42c8-8648-6986c0b9de2f None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:54.251199 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61846bd0-6e78-42c8-8648-6986c0b9de2f None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:54.252676 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61846bd0-6e78-42c8-8648-6986c0b9de2f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:54.260135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 43/85] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:37:54 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:54.260135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61846bd0-6e78-42c8-8648-6986c0b9de2f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:54.270834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:54.313378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1321c-be25-4803-94d2-a8b855efe514 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:54.348146 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8bbbc-1a06-43b6-b88f-100d3668c5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:54.348714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8bbbc-1a06-43b6-b88f-100d3668c5eb None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:54.349003 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8bbbc-1a06-43b6-b88f-100d3668c5eb None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:54.349282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8bbbc-1a06-43b6-b88f-100d3668c5eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:54.351144 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:54.366973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61846bd0-6e78-42c8-8648-6986c0b9de2f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:54.405172 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbf3bf-9458-49f0-b450-39125056cc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:54.405832 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbf3bf-9458-49f0-b450-39125056cc1f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:54.406121 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbf3bf-9458-49f0-b450-39125056cc1f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:54.406407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbf3bf-9458-49f0-b450-39125056cc1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:54.425169 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8bbbc-1a06-43b6-b88f-100d3668c5eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:54.426645 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8bbbc-1a06-43b6-b88f-100d3668c5eb None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:54.485255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbf3bf-9458-49f0-b450-39125056cc1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:54.486721 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbf3bf-9458-49f0-b450-39125056cc1f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:54.492811 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:54.536615 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 43/86] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:37:54.537354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8bbbc-1a06-43b6-b88f-100d3668c5eb None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:54.570888 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:54.589351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 44/87] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:37:54.590192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbf3bf-9458-49f0-b450-39125056cc1f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:54.647976 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8bbbc-1a06-43b6-b88f-100d3668c5eb None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:54.683148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176d9ed-a050-4246-a3e1-8381345308e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:54.683846 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176d9ed-a050-4246-a3e1-8381345308e8 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:54.684138 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176d9ed-a050-4246-a3e1-8381345308e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:54.684425 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176d9ed-a050-4246-a3e1-8381345308e8 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:54.685147 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176d9ed-a050-4246-a3e1-8381345308e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:54.686611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176d9ed-a050-4246-a3e1-8381345308e8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:54.691732 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbf3bf-9458-49f0-b450-39125056cc1f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:54.706397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 44/88] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:37:54 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:37:54.707216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176d9ed-a050-4246-a3e1-8381345308e8 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:54.730167 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92f82e-67f3-42c8-a75a-73c1573fb938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:54.731092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92f82e-67f3-42c8-a75a-73c1573fb938 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:54.731376 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92f82e-67f3-42c8-a75a-73c1573fb938 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:54.731656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92f82e-67f3-42c8-a75a-73c1573fb938 None admin] PATH_INFO: `/v3/users/805b77c8bb67450db935957c46dc8b9b` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:54.733037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92f82e-67f3-42c8-a75a-73c1573fb938 None admin] RBAC: Authorizing `identity:get_user(user_id=805b77c8bb67450db935957c46dc8b9b)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:54.734541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92f82e-67f3-42c8-a75a-73c1573fb938 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:54.736774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 45/89] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:37:54 2022] GET /identity/v3/users/805b77c8bb67450db935957c46dc8b9b => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:54.737617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92f82e-67f3-42c8-a75a-73c1573fb938 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:54.738083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:54.803588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:54.842239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176d9ed-a050-4246-a3e1-8381345308e8 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:54.869335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92f82e-67f3-42c8-a75a-73c1573fb938 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:54.890129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab516c-44ec-4c80-bbe4-6da9b3ddc30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:54.891946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab516c-44ec-4c80-bbe4-6da9b3ddc30d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:54.891946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab516c-44ec-4c80-bbe4-6da9b3ddc30d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:54.891946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab516c-44ec-4c80-bbe4-6da9b3ddc30d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:54.892749 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab516c-44ec-4c80-bbe4-6da9b3ddc30d None admin] RBAC: Adding query filter params (role.id=22f9190503114c4aba5e898e7a460546, scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823, user.id=805b77c8bb67450db935957c46dc8b9b) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:54.893160 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab516c-44ec-4c80-bbe4-6da9b3ddc30d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:54.894768 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab516c-44ec-4c80-bbe4-6da9b3ddc30d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:54.902416 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 45/90] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:37:54 2022] GET /identity/v3/role_assignments?user.id=805b77c8bb67450db935957c46dc8b9b&scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823&role.id=22f9190503114c4aba5e898e7a460546 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:54.902970 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab516c-44ec-4c80-bbe4-6da9b3ddc30d None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:54.932849 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01466e-71ae-4633-b198-ff3ce4844c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:54.933754 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01466e-71ae-4633-b198-ff3ce4844c25 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:54.934044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01466e-71ae-4633-b198-ff3ce4844c25 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:54.934334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01466e-71ae-4633-b198-ff3ce4844c25 None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:54.935632 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01466e-71ae-4633-b198-ff3ce4844c25 None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:54.937164 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01466e-71ae-4633-b198-ff3ce4844c25 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:54.947833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 46/91] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:37:54 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:54.947833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01466e-71ae-4633-b198-ff3ce4844c25 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:55.042287 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:55.076488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab516c-44ec-4c80-bbe4-6da9b3ddc30d None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:55.119449 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94f1c8-a348-4083-9d21-490a76a9886b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:55.120355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94f1c8-a348-4083-9d21-490a76a9886b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:55.120647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94f1c8-a348-4083-9d21-490a76a9886b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:55.120930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94f1c8-a348-4083-9d21-490a76a9886b None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:55.122165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da94f1c8-a348-4083-9d21-490a76a9886b None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:55.124281 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da94f1c8-a348-4083-9d21-490a76a9886b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:55.125621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 46/92] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:37:54 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:55.126231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94f1c8-a348-4083-9d21-490a76a9886b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:55.139541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01466e-71ae-4633-b198-ff3ce4844c25 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:55.181759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:55.207245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b5f2d-0631-4861-827f-5b1a63e12a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:55.208364 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b5f2d-0631-4861-827f-5b1a63e12a6f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:55.208804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b5f2d-0631-4861-827f-5b1a63e12a6f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:55.209294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b5f2d-0631-4861-827f-5b1a63e12a6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:55.279243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94f1c8-a348-4083-9d21-490a76a9886b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:55.324784 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79e5dd-c9ba-4043-a85f-74d0ff901e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:55.326096 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79e5dd-c9ba-4043-a85f-74d0ff901e0b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:55.326647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79e5dd-c9ba-4043-a85f-74d0ff901e0b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:55.327149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79e5dd-c9ba-4043-a85f-74d0ff901e0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:55.328246 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79e5dd-c9ba-4043-a85f-74d0ff901e0b None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823, user.id=5ca745bbd8294b9eaae8eb567675a772) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:55.328785 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79e5dd-c9ba-4043-a85f-74d0ff901e0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:55.330767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79e5dd-c9ba-4043-a85f-74d0ff901e0b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:55.335583 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b5f2d-0631-4861-827f-5b1a63e12a6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:55.338521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 47/93] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:37:55 2022] GET /identity/v3/role_assignments?user.id=5ca745bbd8294b9eaae8eb567675a772&scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:55.339729 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79e5dd-c9ba-4043-a85f-74d0ff901e0b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:55.340276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b5f2d-0631-4861-827f-5b1a63e12a6f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:55.374965 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:55.509677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 47/94] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:37:55.517276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b5f2d-0631-4861-827f-5b1a63e12a6f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:55.534509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79e5dd-c9ba-4043-a85f-74d0ff901e0b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:55.587245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:55.620591 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57a672-120b-4fd4-a47e-1e3daf2ec4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:55.621287 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57a672-120b-4fd4-a47e-1e3daf2ec4af None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:55.621581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57a672-120b-4fd4-a47e-1e3daf2ec4af None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:55.621868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57a672-120b-4fd4-a47e-1e3daf2ec4af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:55.622737 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57a672-120b-4fd4-a47e-1e3daf2ec4af None admin] RBAC: Adding query filter params (role.id=04597f92ac654fb4ac68dbb783d3dc31, scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823, user.id=805b77c8bb67450db935957c46dc8b9b) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:55.623049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57a672-120b-4fd4-a47e-1e3daf2ec4af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:55.625080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57a672-120b-4fd4-a47e-1e3daf2ec4af None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:55.641919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 48/95] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:37:55 2022] GET /identity/v3/role_assignments?user.id=805b77c8bb67450db935957c46dc8b9b&scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823&role.id=04597f92ac654fb4ac68dbb783d3dc31 => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:55.744031 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b5f2d-0631-4861-827f-5b1a63e12a6f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:55.820105 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35215235-e3b4-48b2-8c05-ed9382b6b604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:55.822285 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35215235-e3b4-48b2-8c05-ed9382b6b604 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:55.823069 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35215235-e3b4-48b2-8c05-ed9382b6b604 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:55.823889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35215235-e3b4-48b2-8c05-ed9382b6b604 None admin] PATH_INFO: `/v3/projects/fee9d514657246fe80d694402f1870b0` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:55.826436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35215235-e3b4-48b2-8c05-ed9382b6b604 None admin] RBAC: Authorizing `identity:get_project(project_id=fee9d514657246fe80d694402f1870b0)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:55.829395 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35215235-e3b4-48b2-8c05-ed9382b6b604 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:55.833111 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 48/96] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:37:55 2022] GET /identity/v3/projects/fee9d514657246fe80d694402f1870b0 => generated 312 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:55.839879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57a672-120b-4fd4-a47e-1e3daf2ec4af None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:55.952055 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:55.980464 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57a672-120b-4fd4-a47e-1e3daf2ec4af None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:56.022887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db1c22-006e-4ad3-8a2b-88e176aa09e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:56.023701 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db1c22-006e-4ad3-8a2b-88e176aa09e6 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:56.024018 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db1c22-006e-4ad3-8a2b-88e176aa09e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:56.024302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db1c22-006e-4ad3-8a2b-88e176aa09e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:56.025295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db1c22-006e-4ad3-8a2b-88e176aa09e6 None admin] RBAC: Adding query filter params (role.id=22f9190503114c4aba5e898e7a460546, scope.project.id=fee9d514657246fe80d694402f1870b0, user.id=805b77c8bb67450db935957c46dc8b9b) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:56.025615 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db1c22-006e-4ad3-8a2b-88e176aa09e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:56.027108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db1c22-006e-4ad3-8a2b-88e176aa09e6 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:56.033440 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 49/97] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:37:55 2022] GET /identity/v3/role_assignments?user.id=805b77c8bb67450db935957c46dc8b9b&scope.project.id=fee9d514657246fe80d694402f1870b0&role.id=22f9190503114c4aba5e898e7a460546 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:57.826848 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d449580-4975-433c-aa4e-2c52b62c4d1f None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:57.828352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d449580-4975-433c-aa4e-2c52b62c4d1f None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:57.829178 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d449580-4975-433c-aa4e-2c52b62c4d1f None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:57.831957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 49/98] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:57.837971 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ef16d-6262-4086-a4ed-2f6b0755ebd9 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:57.838667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ef16d-6262-4086-a4ed-2f6b0755ebd9 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:57.838667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ef16d-6262-4086-a4ed-2f6b0755ebd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:57.864783 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c23ef16d-6262-4086-a4ed-2f6b0755ebd9 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:57.866026 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c23ef16d-6262-4086-a4ed-2f6b0755ebd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:57.958304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 50/99] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:37:58.002630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d449580-4975-433c-aa4e-2c52b62c4d1f None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:58.152233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:58.163440 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d449580-4975-433c-aa4e-2c52b62c4d1f None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:58.211336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237e445-b22a-4fce-b7f3-552b87f4dc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:58.212724 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237e445-b22a-4fce-b7f3-552b87f4dc67 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:58.213171 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237e445-b22a-4fce-b7f3-552b87f4dc67 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:58.213693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237e445-b22a-4fce-b7f3-552b87f4dc67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:58.220391 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bb632-0bd7-4778-ad0a-67010fb4eb1a None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:58.220697 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bb632-0bd7-4778-ad0a-67010fb4eb1a None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:58.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bb632-0bd7-4778-ad0a-67010fb4eb1a None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:58.222359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 51/100] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:58.228209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9e5bc-1c95-4f37-83f4-3fee605ec065 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:58.228493 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9e5bc-1c95-4f37-83f4-3fee605ec065 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:58.228777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9e5bc-1c95-4f37-83f4-3fee605ec065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:58.240912 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237e445-b22a-4fce-b7f3-552b87f4dc67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:37:58.246723 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d8c9e5bc-1c95-4f37-83f4-3fee605ec065 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:58.247965 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d8c9e5bc-1c95-4f37-83f4-3fee605ec065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-7237e445-b22a-4fce-b7f3-552b87f4dc67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:37:58.249739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:37:58.250641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:37:58.251431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:37:58.251431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:37:58.251431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:37:58.251431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:37:58.251431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:37:58.251431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:37:58.251431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:37:58.251431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:37:58.251431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:37:58.251431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:37:58.251431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:37:58.251431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:37:58.257336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 50/101] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:37:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:37:58.263066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237e445-b22a-4fce-b7f3-552b87f4dc67 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:58.401868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:58.418209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 52/102] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:37:58.418209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8e3a8-ae0b-4b1e-b393-201a535e7c49 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:58.418456 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8e3a8-ae0b-4b1e-b393-201a535e7c49 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:58.418688 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8e3a8-ae0b-4b1e-b393-201a535e7c49 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:58.425024 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 53/103] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:58.433180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacb345-450e-4147-839f-8b276c2ab84c None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:58.433483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacb345-450e-4147-839f-8b276c2ab84c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:58.433757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacb345-450e-4147-839f-8b276c2ab84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:58.434833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237e445-b22a-4fce-b7f3-552b87f4dc67 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:58.467744 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-2dacb345-450e-4147-839f-8b276c2ab84c None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:58.469015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-2dacb345-450e-4147-839f-8b276c2ab84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:58.481643 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bfb48e-b339-4a43-addc-4a1ee44cc74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:58.482696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bfb48e-b339-4a43-addc-4a1ee44cc74f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:58.483208 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bfb48e-b339-4a43-addc-4a1ee44cc74f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:58.483701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bfb48e-b339-4a43-addc-4a1ee44cc74f None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:58.484710 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bfb48e-b339-4a43-addc-4a1ee44cc74f None admin] RBAC: Adding query filter params (name=member) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:58.485308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bfb48e-b339-4a43-addc-4a1ee44cc74f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:58.487053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bfb48e-b339-4a43-addc-4a1ee44cc74f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:58.497649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 51/104] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:37:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:58.498836 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bfb48e-b339-4a43-addc-4a1ee44cc74f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:58.516145 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:58.622935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bfb48e-b339-4a43-addc-4a1ee44cc74f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:58.625242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 54/105] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:37:58.627002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacb345-450e-4147-839f-8b276c2ab84c None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:58.668583 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4267544-e955-4c38-8719-af96f763394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:58.669989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4267544-e955-4c38-8719-af96f763394a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:58.671045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4267544-e955-4c38-8719-af96f763394a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:58.671548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4267544-e955-4c38-8719-af96f763394a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:58.673437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4267544-e955-4c38-8719-af96f763394a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:58.676004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4267544-e955-4c38-8719-af96f763394a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:58.678253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 52/106] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:37:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:58.680924 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be42278-3440-4133-99fa-e324504c02fe None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:58.681910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be42278-3440-4133-99fa-e324504c02fe None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:58.682423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be42278-3440-4133-99fa-e324504c02fe None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:58.684233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 53/107] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:58.685179 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be42278-3440-4133-99fa-e324504c02fe None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:58.783436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacb345-450e-4147-839f-8b276c2ab84c None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:58.823300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:58.827315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:58.839050 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eff858-9971-4088-8fe0-83d45909de41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:58.840028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff858-9971-4088-8fe0-83d45909de41 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:58.840587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff858-9971-4088-8fe0-83d45909de41 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:58.840929 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff858-9971-4088-8fe0-83d45909de41 None admin] PATH_INFO: `/v3/users/805b77c8bb67450db935957c46dc8b9b` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:58.842355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eff858-9971-4088-8fe0-83d45909de41 None admin] RBAC: Authorizing `identity:get_user(user_id=805b77c8bb67450db935957c46dc8b9b)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:58.844472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eff858-9971-4088-8fe0-83d45909de41 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:58.850708 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 55/108] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:37:58 2022] GET /identity/v3/users/805b77c8bb67450db935957c46dc8b9b => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:58.850708 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eff858-9971-4088-8fe0-83d45909de41 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:58.856880 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be42278-3440-4133-99fa-e324504c02fe None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:58.901300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75867b7d-85de-4867-b0ed-3d4d8948262a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:58.902256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75867b7d-85de-4867-b0ed-3d4d8948262a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:58.902603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75867b7d-85de-4867-b0ed-3d4d8948262a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:58.902892 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75867b7d-85de-4867-b0ed-3d4d8948262a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:58.915912 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75867b7d-85de-4867-b0ed-3d4d8948262a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-75867b7d-85de-4867-b0ed-3d4d8948262a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:37:58.924549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:37:58.925411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:37:58.926150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:37:58.926150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:37:58.926150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:37:58.926150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:37:58.926150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:37:58.926150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:37:58.926150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:37:58.926150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:37:58.926150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:37:58.926150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:37:58.926150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:37:58.926150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:37:58.926150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 54/109] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:37:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:37:58.927494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86a43b-96e6-401c-849d-e9b93147b577 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:58.927494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86a43b-96e6-401c-849d-e9b93147b577 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:58.927828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86a43b-96e6-401c-849d-e9b93147b577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:58.947910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-aa86a43b-96e6-401c-849d-e9b93147b577 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:58.949258 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-aa86a43b-96e6-401c-849d-e9b93147b577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:58.968712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eff858-9971-4088-8fe0-83d45909de41 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:59.018098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1366490-16d3-41af-8b7a-b88b53c58b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:59.018868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1366490-16d3-41af-8b7a-b88b53c58b06 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:59.019230 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1366490-16d3-41af-8b7a-b88b53c58b06 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:59.019576 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1366490-16d3-41af-8b7a-b88b53c58b06 None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:59.020399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1366490-16d3-41af-8b7a-b88b53c58b06 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:59.021939 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1366490-16d3-41af-8b7a-b88b53c58b06 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:59.028988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 55/110] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:37:59.040765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86a43b-96e6-401c-849d-e9b93147b577 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:59.058610 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 56/111] 10.222.0.60 () {60 vars in 1141 bytes} [Thu Jul 21 20:37:58 2022] POST /identity/v3/projects => generated 302 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:37:59.058610 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1366490-16d3-41af-8b7a-b88b53c58b06 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:59.115581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:59.141192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:59.177010 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1366490-16d3-41af-8b7a-b88b53c58b06 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:59.177256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86a43b-96e6-401c-849d-e9b93147b577 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:59.216193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2e2e3-d8e9-4f5b-a809-c4555b9ccf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:59.216942 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2e2e3-d8e9-4f5b-a809-c4555b9ccf9e None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:59.217664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2e2e3-d8e9-4f5b-a809-c4555b9ccf9e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:59.217664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2e2e3-d8e9-4f5b-a809-c4555b9ccf9e None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:59.218359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2e2e3-d8e9-4f5b-a809-c4555b9ccf9e None admin] RBAC: Adding query filter params (name=reader) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:59.218695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed99dd-b12d-4d1c-bf2d-eada33f97994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:59.219018 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2e2e3-d8e9-4f5b-a809-c4555b9ccf9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:59.219944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed99dd-b12d-4d1c-bf2d-eada33f97994 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:59.220377 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed99dd-b12d-4d1c-bf2d-eada33f97994 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:59.220518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2e2e3-d8e9-4f5b-a809-c4555b9ccf9e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:59.220941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed99dd-b12d-4d1c-bf2d-eada33f97994 None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:59.222487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed99dd-b12d-4d1c-bf2d-eada33f97994 None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:59.224365 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed99dd-b12d-4d1c-bf2d-eada33f97994 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:59.227634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 56/112] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:37:59 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:59.229762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c6399-1732-4826-abfe-dbfa54e07e04 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:59.230213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c6399-1732-4826-abfe-dbfa54e07e04 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:59.230648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c6399-1732-4826-abfe-dbfa54e07e04 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:59.232204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 57/113] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:59.233294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c6399-1732-4826-abfe-dbfa54e07e04 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:59.239583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 57/114] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:37:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:59.240459 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2e2e3-d8e9-4f5b-a809-c4555b9ccf9e None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:59.357909 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2e2e3-d8e9-4f5b-a809-c4555b9ccf9e None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:59.374464 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:59.378285 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c6399-1732-4826-abfe-dbfa54e07e04 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:59.411736 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce99b5-d2b8-4546-b646-56968b47e9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:59.411736 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce99b5-d2b8-4546-b646-56968b47e9e0 None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:59.411736 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce99b5-d2b8-4546-b646-56968b47e9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:59.411736 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce99b5-d2b8-4546-b646-56968b47e9e0 None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823/users/805b77c8bb67450db935957c46dc8b9b/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:59.414819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce99b5-d2b8-4546-b646-56968b47e9e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=32ebd8e81f314e8fa20cb9ba0042a823, user_id=805b77c8bb67450db935957c46dc8b9b, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:59.417979 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce99b5-d2b8-4546-b646-56968b47e9e0 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:59.427674 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b52aed-3943-4370-930b-ad73db80edcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:59.428549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b52aed-3943-4370-930b-ad73db80edcc None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:59.428908 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b52aed-3943-4370-930b-ad73db80edcc None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:59.429265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b52aed-3943-4370-930b-ad73db80edcc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:59.436797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 58/115] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:37:59 2022] PUT /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823/users/805b77c8bb67450db935957c46dc8b9b/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:37:59.438903 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:59.439309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2380bf-f0c6-4166-9704-991142b1c012 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:59.439629 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2380bf-f0c6-4166-9704-991142b1c012 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:59.440021 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2380bf-f0c6-4166-9704-991142b1c012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:59.446003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b52aed-3943-4370-930b-ad73db80edcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-05b52aed-3943-4370-930b-ad73db80edcc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:37:59.457358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:37:59.458265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:37:59.458999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:37:59.458999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:37:59.458999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:37:59.458999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:37:59.458999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:37:59.458999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:37:59.458999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:37:59.458999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:37:59.458999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:37:59.458999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:37:59.458999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:37:59.458999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:37:59.459556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 58/116] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:37:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:37:59.463749 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b52aed-3943-4370-930b-ad73db80edcc None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:59.467207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-9f2380bf-f0c6-4166-9704-991142b1c012 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:59.468642 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-9f2380bf-f0c6-4166-9704-991142b1c012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:59.572158 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 59/117] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:37:59.573888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339cc62-9e09-4cb9-969e-b73b5110cf55 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:59.574412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339cc62-9e09-4cb9-969e-b73b5110cf55 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:59.574911 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339cc62-9e09-4cb9-969e-b73b5110cf55 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:59.576512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 60/118] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:37:59.577582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339cc62-9e09-4cb9-969e-b73b5110cf55 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:59.606697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b52aed-3943-4370-930b-ad73db80edcc None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:59.656956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051ebb8-e5e3-487e-820e-ad9883e65ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:59.657849 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051ebb8-e5e3-487e-820e-ad9883e65ba4 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:59.658133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051ebb8-e5e3-487e-820e-ad9883e65ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:59.658408 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051ebb8-e5e3-487e-820e-ad9883e65ba4 None admin] PATH_INFO: `/v3/users/80b704353f364eb798f11553af1fd8a5` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:59.676508 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051ebb8-e5e3-487e-820e-ad9883e65ba4 None admin] RBAC: Authorizing `identity:get_user(user_id=80b704353f364eb798f11553af1fd8a5)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:59.678043 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051ebb8-e5e3-487e-820e-ad9883e65ba4 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:59.679851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 59/119] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:37:59 2022] GET /identity/v3/users/80b704353f364eb798f11553af1fd8a5 => generated 289 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:59.681081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204716e-81fb-4b1c-8237-fb5faad1cae9 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:59.681367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204716e-81fb-4b1c-8237-fb5faad1cae9 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:59.681651 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204716e-81fb-4b1c-8237-fb5faad1cae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:59.704032 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-a204716e-81fb-4b1c-8237-fb5faad1cae9 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:37:59.705224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a204716e-81fb-4b1c-8237-fb5faad1cae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:37:59.717244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339cc62-9e09-4cb9-969e-b73b5110cf55 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:59.720998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:59.759293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:37:59.777529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 60/120] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:37:59.778386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204716e-81fb-4b1c-8237-fb5faad1cae9 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:59.798469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580ff1b-fc2a-4a07-bf03-765798083299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:59.799139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580ff1b-fc2a-4a07-bf03-765798083299 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:59.799427 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580ff1b-fc2a-4a07-bf03-765798083299 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:59.799705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580ff1b-fc2a-4a07-bf03-765798083299 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:59.800507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580ff1b-fc2a-4a07-bf03-765798083299 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:37:59.800815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580ff1b-fc2a-4a07-bf03-765798083299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:37:59.802244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580ff1b-fc2a-4a07-bf03-765798083299 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:37:59.823170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 61/121] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:37:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:37:59.824692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580ff1b-fc2a-4a07-bf03-765798083299 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:37:59.915990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204716e-81fb-4b1c-8237-fb5faad1cae9 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:59.965750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d34f8-b57f-472d-a99f-32fe47f8613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:37:59.966900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d34f8-b57f-472d-a99f-32fe47f8613b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:37:59.967457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d34f8-b57f-472d-a99f-32fe47f8613b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:37:59.967969 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d34f8-b57f-472d-a99f-32fe47f8613b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:37:59.968836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580ff1b-fc2a-4a07-bf03-765798083299 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:37:59.989177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d34f8-b57f-472d-a99f-32fe47f8613b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-399d34f8-b57f-472d-a99f-32fe47f8613b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:00.000962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:00.001872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:00.002593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:00.002593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:00.002593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:00.002593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:00.002593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:00.002593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:00.002593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:00.002593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:00.002593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:00.002593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:00.002593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:00.002593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:00.002593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 61/122] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:37:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:00.003285 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d34f8-b57f-472d-a99f-32fe47f8613b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:00.012217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9d572-e4d6-4e94-9974-79f96d441c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:00.013210 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9d572-e4d6-4e94-9974-79f96d441c9a None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:00.013690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9d572-e4d6-4e94-9974-79f96d441c9a None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:00.014188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9d572-e4d6-4e94-9974-79f96d441c9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:00.058976 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:00.069640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:00.111019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9d572-e4d6-4e94-9974-79f96d441c9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:00.113152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9d572-e4d6-4e94-9974-79f96d441c9a None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:00.136302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d34f8-b57f-472d-a99f-32fe47f8613b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:00.183347 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e15460-1d16-4695-a5ed-785ac4e77932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:00.184316 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e15460-1d16-4695-a5ed-785ac4e77932 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:00.184608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e15460-1d16-4695-a5ed-785ac4e77932 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:00.184889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e15460-1d16-4695-a5ed-785ac4e77932 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:00.199616 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e15460-1d16-4695-a5ed-785ac4e77932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-16e15460-1d16-4695-a5ed-785ac4e77932 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:00.207783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:00.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:00.209414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:00.209414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:00.209414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:00.209414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:00.209414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:00.209414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:00.209414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:00.209414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:00.209414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:00.209414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:00.209414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:00.209414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:00.214648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 62/123] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:00.215524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e15460-1d16-4695-a5ed-785ac4e77932 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:00.235954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 62/124] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:37:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:00.237190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9d572-e4d6-4e94-9974-79f96d441c9a None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:00.305853 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:00.336718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e15460-1d16-4695-a5ed-785ac4e77932 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:00.344392 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:00.349985 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9d572-e4d6-4e94-9974-79f96d441c9a None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:00.376679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd73c52-b812-4577-b7fa-85b2a444d87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:00.377593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd73c52-b812-4577-b7fa-85b2a444d87f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:00.377879 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd73c52-b812-4577-b7fa-85b2a444d87f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:00.378167 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd73c52-b812-4577-b7fa-85b2a444d87f None admin] PATH_INFO: `/v3/users/5ca745bbd8294b9eaae8eb567675a772` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:00.379378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd73c52-b812-4577-b7fa-85b2a444d87f None admin] RBAC: Authorizing `identity:get_user(user_id=5ca745bbd8294b9eaae8eb567675a772)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:00.381060 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd73c52-b812-4577-b7fa-85b2a444d87f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:00.383101 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 63/125] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:00 2022] GET /identity/v3/users/5ca745bbd8294b9eaae8eb567675a772 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:00.384101 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd73c52-b812-4577-b7fa-85b2a444d87f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:00.390137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4f046-ca67-4f0c-b083-9fbc5d3be8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:00.391150 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4f046-ca67-4f0c-b083-9fbc5d3be8a0 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:00.391637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4f046-ca67-4f0c-b083-9fbc5d3be8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:00.392175 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4f046-ca67-4f0c-b083-9fbc5d3be8a0 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:00.393148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4f046-ca67-4f0c-b083-9fbc5d3be8a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:00.393667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4f046-ca67-4f0c-b083-9fbc5d3be8a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:00.395257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4f046-ca67-4f0c-b083-9fbc5d3be8a0 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:00.410596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 63/126] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:00.411818 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4f046-ca67-4f0c-b083-9fbc5d3be8a0 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:00.496980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd73c52-b812-4577-b7fa-85b2a444d87f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:00.529045 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4f046-ca67-4f0c-b083-9fbc5d3be8a0 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:00.534633 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e9313-cf10-49b0-bf50-5b8d334471d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:00.535315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e9313-cf10-49b0-bf50-5b8d334471d1 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:00.535601 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e9313-cf10-49b0-bf50-5b8d334471d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:00.535909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e9313-cf10-49b0-bf50-5b8d334471d1 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:00.536665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e9313-cf10-49b0-bf50-5b8d334471d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:00.536972 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e9313-cf10-49b0-bf50-5b8d334471d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:00.538485 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e9313-cf10-49b0-bf50-5b8d334471d1 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:00.552674 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 64/127] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:00.553279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e9313-cf10-49b0-bf50-5b8d334471d1 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:00.559626 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:00.572431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2ec7c-6fd5-48af-8fe4-19db8eb78a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:00.573703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2ec7c-6fd5-48af-8fe4-19db8eb78a95 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:00.574193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2ec7c-6fd5-48af-8fe4-19db8eb78a95 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:00.574672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2ec7c-6fd5-48af-8fe4-19db8eb78a95 None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:00.576391 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2ec7c-6fd5-48af-8fe4-19db8eb78a95 None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:00.578244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2ec7c-6fd5-48af-8fe4-19db8eb78a95 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:00.580449 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 64/128] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:00 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:00.581444 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2ec7c-6fd5-48af-8fe4-19db8eb78a95 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:00.598155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:00.690903 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e9313-cf10-49b0-bf50-5b8d334471d1 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:00.702093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2ec7c-6fd5-48af-8fe4-19db8eb78a95 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:00.739942 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a77ed-3b64-44a3-acc1-f8a1ec79e6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:00.741211 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a77ed-3b64-44a3-acc1-f8a1ec79e6ba None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:00.741702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a77ed-3b64-44a3-acc1-f8a1ec79e6ba None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:00.742430 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a77ed-3b64-44a3-acc1-f8a1ec79e6ba None admin] PATH_INFO: `/v3/users/805b77c8bb67450db935957c46dc8b9b` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:00.744135 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a77ed-3b64-44a3-acc1-f8a1ec79e6ba None admin] RBAC: Authorizing `identity:get_user(user_id=805b77c8bb67450db935957c46dc8b9b)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:00.745151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c5978-0bb6-4866-a362-82fedfe7e346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:00.746031 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c5978-0bb6-4866-a362-82fedfe7e346 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:00.746317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c5978-0bb6-4866-a362-82fedfe7e346 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:00.746428 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a77ed-3b64-44a3-acc1-f8a1ec79e6ba None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:00.746586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c5978-0bb6-4866-a362-82fedfe7e346 None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:00.747937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c5978-0bb6-4866-a362-82fedfe7e346 None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:00.748870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 65/129] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:00 2022] GET /identity/v3/users/805b77c8bb67450db935957c46dc8b9b => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:00.749447 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c5978-0bb6-4866-a362-82fedfe7e346 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:00.750144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a77ed-3b64-44a3-acc1-f8a1ec79e6ba None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:00.751316 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 65/130] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:00 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:00.752110 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c5978-0bb6-4866-a362-82fedfe7e346 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:00.828689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:00.862378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c5978-0bb6-4866-a362-82fedfe7e346 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:00.864785 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:00.880422 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a77ed-3b64-44a3-acc1-f8a1ec79e6ba None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:00.901847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcb16e-82b3-44df-8a0b-572ebb169fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:00.902519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcb16e-82b3-44df-8a0b-572ebb169fc9 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:00.902806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcb16e-82b3-44df-8a0b-572ebb169fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:00.903085 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcb16e-82b3-44df-8a0b-572ebb169fc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:00.903953 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcb16e-82b3-44df-8a0b-572ebb169fc9 None admin] RBAC: Adding query filter params (role.id=1052f958732d4259a0658b40d1f9c193, scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823, user.id=80b704353f364eb798f11553af1fd8a5) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:00.904268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcb16e-82b3-44df-8a0b-572ebb169fc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:00.906558 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcb16e-82b3-44df-8a0b-572ebb169fc9 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:00.915142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 66/131] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:00 2022] GET /identity/v3/role_assignments?user.id=80b704353f364eb798f11553af1fd8a5&scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823&role.id=1052f958732d4259a0658b40d1f9c193 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:00.916600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcb16e-82b3-44df-8a0b-572ebb169fc9 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:00.952339 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d4cb0-1bad-420f-b3d2-fb0551d33b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:00.954393 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d4cb0-1bad-420f-b3d2-fb0551d33b31 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:00.955212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d4cb0-1bad-420f-b3d2-fb0551d33b31 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:00.956039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d4cb0-1bad-420f-b3d2-fb0551d33b31 None admin] PATH_INFO: `/v3/users/805b77c8bb67450db935957c46dc8b9b` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:00.959268 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d4cb0-1bad-420f-b3d2-fb0551d33b31 None admin] RBAC: Authorizing `identity:get_user(user_id=805b77c8bb67450db935957c46dc8b9b)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:00.962977 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d4cb0-1bad-420f-b3d2-fb0551d33b31 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:00.970546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 66/132] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:00 2022] GET /identity/v3/users/805b77c8bb67450db935957c46dc8b9b => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:00.973039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d4cb0-1bad-420f-b3d2-fb0551d33b31 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:01.119077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcb16e-82b3-44df-8a0b-572ebb169fc9 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:01.160757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc515df0-44fa-4f1d-9b17-cab8df8a473a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:01.161729 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc515df0-44fa-4f1d-9b17-cab8df8a473a None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:01.162044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc515df0-44fa-4f1d-9b17-cab8df8a473a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:01.162335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc515df0-44fa-4f1d-9b17-cab8df8a473a None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823/users/5ca745bbd8294b9eaae8eb567675a772/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:01.165361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc515df0-44fa-4f1d-9b17-cab8df8a473a None admin] RBAC: Authorizing `identity:create_grant(project_id=32ebd8e81f314e8fa20cb9ba0042a823, user_id=5ca745bbd8294b9eaae8eb567675a772, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:01.168120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc515df0-44fa-4f1d-9b17-cab8df8a473a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:01.188698 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 67/133] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:38:00 2022] PUT /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823/users/5ca745bbd8294b9eaae8eb567675a772/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:38:01.189451 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc515df0-44fa-4f1d-9b17-cab8df8a473a None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:01.225973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:01.281131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d4cb0-1bad-420f-b3d2-fb0551d33b31 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:01.316216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc515df0-44fa-4f1d-9b17-cab8df8a473a None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:01.343330 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:01.355938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bd4e6-ed53-46b3-9caf-ecf22ffb41ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:01.357061 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bd4e6-ed53-46b3-9caf-ecf22ffb41ca None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:01.357478 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bd4e6-ed53-46b3-9caf-ecf22ffb41ca None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:01.357923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bd4e6-ed53-46b3-9caf-ecf22ffb41ca None admin] PATH_INFO: `/v3/projects/fee9d514657246fe80d694402f1870b0` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:01.359856 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bd4e6-ed53-46b3-9caf-ecf22ffb41ca None admin] RBAC: Authorizing `identity:get_project(project_id=fee9d514657246fe80d694402f1870b0)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:01.361900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bd4e6-ed53-46b3-9caf-ecf22ffb41ca None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:01.364781 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c43e28-78e3-44ea-891e-8dbdc1acd2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:01.365877 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c43e28-78e3-44ea-891e-8dbdc1acd2da None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:01.366926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 68/134] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:01 2022] GET /identity/v3/projects/fee9d514657246fe80d694402f1870b0 => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:01.367521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c43e28-78e3-44ea-891e-8dbdc1acd2da None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:01.367969 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c43e28-78e3-44ea-891e-8dbdc1acd2da None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:01.369580 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c43e28-78e3-44ea-891e-8dbdc1acd2da None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:01.371208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c43e28-78e3-44ea-891e-8dbdc1acd2da None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:01.373147 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bd4e6-ed53-46b3-9caf-ecf22ffb41ca None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:01.383042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 67/135] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:00 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:01.384414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c43e28-78e3-44ea-891e-8dbdc1acd2da None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:01.576431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:01.587113 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bd4e6-ed53-46b3-9caf-ecf22ffb41ca None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:01.606591 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c43e28-78e3-44ea-891e-8dbdc1acd2da None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:01.644367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d88c7-ca82-4d20-9bc3-ef3928231ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:01.645342 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d88c7-ca82-4d20-9bc3-ef3928231ee4 None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:01.645628 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d88c7-ca82-4d20-9bc3-ef3928231ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:01.645903 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d88c7-ca82-4d20-9bc3-ef3928231ee4 None admin] PATH_INFO: `/v3/projects/fee9d514657246fe80d694402f1870b0/users/805b77c8bb67450db935957c46dc8b9b/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:01.654353 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d88c7-ca82-4d20-9bc3-ef3928231ee4 None admin] RBAC: Authorizing `identity:create_grant(project_id=fee9d514657246fe80d694402f1870b0, user_id=805b77c8bb67450db935957c46dc8b9b, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:01.655878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d88c7-ca82-4d20-9bc3-ef3928231ee4 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:01.684267 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8c47a-e8b5-44d3-bac7-5ab7fe05f099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:01.685895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8c47a-e8b5-44d3-bac7-5ab7fe05f099 None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:01.686486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8c47a-e8b5-44d3-bac7-5ab7fe05f099 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:01.686954 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 69/136] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:38:01 2022] PUT /identity/v3/projects/fee9d514657246fe80d694402f1870b0/users/805b77c8bb67450db935957c46dc8b9b/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:38:01.687110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8c47a-e8b5-44d3-bac7-5ab7fe05f099 None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823/users/805b77c8bb67450db935957c46dc8b9b/roles/04597f92ac654fb4ac68dbb783d3dc31` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:01.699329 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e8c47a-e8b5-44d3-bac7-5ab7fe05f099 None admin] RBAC: Authorizing `identity:create_grant(project_id=32ebd8e81f314e8fa20cb9ba0042a823, user_id=805b77c8bb67450db935957c46dc8b9b, role_id=04597f92ac654fb4ac68dbb783d3dc31)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:01.702339 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e8c47a-e8b5-44d3-bac7-5ab7fe05f099 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:01.738297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 68/137] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:38:01 2022] PUT /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823/users/805b77c8bb67450db935957c46dc8b9b/roles/04597f92ac654fb4ac68dbb783d3dc31 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:38:02.092554 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0015e72-7468-4027-8bf8-d4b48c012e8c None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:02.093053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0015e72-7468-4027-8bf8-d4b48c012e8c None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:02.093124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0015e72-7468-4027-8bf8-d4b48c012e8c None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:02.094481 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 70/138] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:02.105207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a91ddb-6b22-4d2c-b038-1ae37d5e834b None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:02.105700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a91ddb-6b22-4d2c-b038-1ae37d5e834b None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:02.105774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a91ddb-6b22-4d2c-b038-1ae37d5e834b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:02.108808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd422f-b468-4047-86ec-8945f6b3cf63 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:02.109098 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd422f-b468-4047-86ec-8945f6b3cf63 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:02.109371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd422f-b468-4047-86ec-8945f6b3cf63 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:02.112583 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 71/139] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:02.117470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38372adf-2b29-4546-a98c-cbf4f8a83b19 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:02.117720 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38372adf-2b29-4546-a98c-cbf4f8a83b19 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:02.118024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38372adf-2b29-4546-a98c-cbf4f8a83b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:02.131657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-69a91ddb-6b22-4d2c-b038-1ae37d5e834b None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:02.133670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-69a91ddb-6b22-4d2c-b038-1ae37d5e834b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:02.144678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-38372adf-2b29-4546-a98c-cbf4f8a83b19 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:02.146654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-38372adf-2b29-4546-a98c-cbf4f8a83b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:02.244557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:02.273919 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 72/140] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:02.277504 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 69/141] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:02.320917 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38372adf-2b29-4546-a98c-cbf4f8a83b19 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:02.323512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a91ddb-6b22-4d2c-b038-1ae37d5e834b None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:02.512982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a91ddb-6b22-4d2c-b038-1ae37d5e834b None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:02.524722 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:02.556392 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89a0d0-be68-4e67-96b7-02db72941261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:02.557672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89a0d0-be68-4e67-96b7-02db72941261 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:02.558158 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89a0d0-be68-4e67-96b7-02db72941261 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:02.558696 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89a0d0-be68-4e67-96b7-02db72941261 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:02.560170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89a0d0-be68-4e67-96b7-02db72941261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:02.561990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89a0d0-be68-4e67-96b7-02db72941261 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:02.564205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 70/142] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:02.570844 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89a0d0-be68-4e67-96b7-02db72941261 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:02.573105 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38372adf-2b29-4546-a98c-cbf4f8a83b19 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:02.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9b4ec-dc15-4fe7-9683-92b7cc9d9f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:02.619320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9b4ec-dc15-4fe7-9683-92b7cc9d9f36 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:02.619320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9b4ec-dc15-4fe7-9683-92b7cc9d9f36 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:02.619578 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9b4ec-dc15-4fe7-9683-92b7cc9d9f36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:02.636128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9b4ec-dc15-4fe7-9683-92b7cc9d9f36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:02.644202 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-2ac9b4ec-dc15-4fe7-9683-92b7cc9d9f36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:02.653327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:02.654193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:02.654935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:02.654935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:02.654935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:02.654935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:02.654935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:02.654935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:02.654935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:02.654935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:02.654935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:02.654935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:02.654935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:02.654935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:02.654935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 73/143] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:02.671086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9b4ec-dc15-4fe7-9683-92b7cc9d9f36 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:02.701126 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89a0d0-be68-4e67-96b7-02db72941261 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:02.743256 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740c9bb-d4be-43b6-baa4-8e841afc42b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:02.744377 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740c9bb-d4be-43b6-baa4-8e841afc42b7 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:02.744899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740c9bb-d4be-43b6-baa4-8e841afc42b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:02.745390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740c9bb-d4be-43b6-baa4-8e841afc42b7 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:02.746367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740c9bb-d4be-43b6-baa4-8e841afc42b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:02.748157 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740c9bb-d4be-43b6-baa4-8e841afc42b7 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:02.787579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 71/144] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:38:02 2022] POST /identity/v3/users => generated 283 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:38:02.866438 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9b4ec-dc15-4fe7-9683-92b7cc9d9f36 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:02.921666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932170f0-8597-4f0c-a653-e6cc6d99f43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:02.922717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932170f0-8597-4f0c-a653-e6cc6d99f43c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:02.923222 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932170f0-8597-4f0c-a653-e6cc6d99f43c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:02.923743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932170f0-8597-4f0c-a653-e6cc6d99f43c None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:02.924818 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932170f0-8597-4f0c-a653-e6cc6d99f43c None admin] RBAC: Adding query filter params (name=member) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:02.925359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932170f0-8597-4f0c-a653-e6cc6d99f43c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:02.927012 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932170f0-8597-4f0c-a653-e6cc6d99f43c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:02.941987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 74/145] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:02.948640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740c9bb-d4be-43b6-baa4-8e841afc42b7 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:03.068358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740c9bb-d4be-43b6-baa4-8e841afc42b7 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:03.088098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:03.109192 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47c610-e904-444d-8cb4-99481b2fcec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:03.110089 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47c610-e904-444d-8cb4-99481b2fcec6 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:03.110375 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47c610-e904-444d-8cb4-99481b2fcec6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:03.110655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47c610-e904-444d-8cb4-99481b2fcec6 None admin] PATH_INFO: `/v3/users/805b77c8bb67450db935957c46dc8b9b` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:03.111921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47c610-e904-444d-8cb4-99481b2fcec6 None admin] RBAC: Authorizing `identity:get_user(user_id=805b77c8bb67450db935957c46dc8b9b)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:03.113523 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47c610-e904-444d-8cb4-99481b2fcec6 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:03.115253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 72/146] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:02 2022] GET /identity/v3/users/805b77c8bb67450db935957c46dc8b9b => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:03.121173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932170f0-8597-4f0c-a653-e6cc6d99f43c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:03.133157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:03.256067 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932170f0-8597-4f0c-a653-e6cc6d99f43c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:03.306047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acd639-b007-4b2c-96ef-fe7380c2cbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:03.307097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acd639-b007-4b2c-96ef-fe7380c2cbfd None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:03.307607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acd639-b007-4b2c-96ef-fe7380c2cbfd None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:03.308105 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acd639-b007-4b2c-96ef-fe7380c2cbfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:03.399859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acd639-b007-4b2c-96ef-fe7380c2cbfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:03.401751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acd639-b007-4b2c-96ef-fe7380c2cbfd None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:03.416970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:03.533679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 75/147] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:03.539640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47c610-e904-444d-8cb4-99481b2fcec6 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:03.664382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47c610-e904-444d-8cb4-99481b2fcec6 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:03.703493 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a847cd-b762-4c97-a857-c29cd53831a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:03.704747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a847cd-b762-4c97-a857-c29cd53831a7 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:03.705215 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a847cd-b762-4c97-a857-c29cd53831a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:03.705666 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a847cd-b762-4c97-a857-c29cd53831a7 None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:03.707194 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a847cd-b762-4c97-a857-c29cd53831a7 None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:03.708958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a847cd-b762-4c97-a857-c29cd53831a7 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:03.711569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 73/148] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:03 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:03.718080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acd639-b007-4b2c-96ef-fe7380c2cbfd None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:03.899789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acd639-b007-4b2c-96ef-fe7380c2cbfd None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:03.918774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:03.958406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec63e4-06ae-49e0-9ca1-d92ead607228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:03.959449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec63e4-06ae-49e0-9ca1-d92ead607228 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:03.959974 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec63e4-06ae-49e0-9ca1-d92ead607228 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:03.960992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec63e4-06ae-49e0-9ca1-d92ead607228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:03.962188 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec63e4-06ae-49e0-9ca1-d92ead607228 None admin] RBAC: Adding query filter params (role.id=22f9190503114c4aba5e898e7a460546, scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823, user.id=805b77c8bb67450db935957c46dc8b9b) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:03.962709 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec63e4-06ae-49e0-9ca1-d92ead607228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:03.964421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec63e4-06ae-49e0-9ca1-d92ead607228 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:03.976567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 76/149] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:03 2022] GET /identity/v3/role_assignments?user.id=805b77c8bb67450db935957c46dc8b9b&scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823&role.id=22f9190503114c4aba5e898e7a460546 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:04.420837 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fad98-3c71-45d4-a6e1-5f723c3fccd4 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:04.421345 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fad98-3c71-45d4-a6e1-5f723c3fccd4 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:04.421420 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fad98-3c71-45d4-a6e1-5f723c3fccd4 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:04.422908 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 74/150] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:04.428453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff733a-3bea-4a3c-b0b5-dfe147411455 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:04.429378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff733a-3bea-4a3c-b0b5-dfe147411455 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:04.430344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff733a-3bea-4a3c-b0b5-dfe147411455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:04.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-73ff733a-3bea-4a3c-b0b5-dfe147411455 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:04.458824 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-73ff733a-3bea-4a3c-b0b5-dfe147411455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:04.540441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 77/151] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:04.571953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fad98-3c71-45d4-a6e1-5f723c3fccd4 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:04.646580 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:04.707070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28deb14e-73a5-4efc-8811-f452cd8aaa05 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:04.707665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28deb14e-73a5-4efc-8811-f452cd8aaa05 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:04.708308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28deb14e-73a5-4efc-8811-f452cd8aaa05 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:04.710050 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 78/152] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:38:04.717282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feec6a8-47f4-4896-ab1d-8d9a542dd996 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:04.717863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feec6a8-47f4-4896-ab1d-8d9a542dd996 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:04.718366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feec6a8-47f4-4896-ab1d-8d9a542dd996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:04.720016 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fad98-3c71-45d4-a6e1-5f723c3fccd4 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:04.738072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-5feec6a8-47f4-4896-ab1d-8d9a542dd996 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:04.739630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-5feec6a8-47f4-4896-ab1d-8d9a542dd996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:04.773984 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96215462-b20b-422e-80f1-83eb629186aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:04.775158 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96215462-b20b-422e-80f1-83eb629186aa None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:04.775747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96215462-b20b-422e-80f1-83eb629186aa None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:04.776297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96215462-b20b-422e-80f1-83eb629186aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:04.802796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96215462-b20b-422e-80f1-83eb629186aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-96215462-b20b-422e-80f1-83eb629186aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:04.819472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:04.820474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:04.821216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:04.821216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:04.821216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:04.821216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:04.821216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:04.821216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:04.821216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:04.821216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:04.821216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:04.821216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:04.821216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:04.821216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:04.823756 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 75/153] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:04.832851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96215462-b20b-422e-80f1-83eb629186aa None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:04.898744 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 79/154] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:38:04.962519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feec6a8-47f4-4896-ab1d-8d9a542dd996 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:05.036591 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96215462-b20b-422e-80f1-83eb629186aa None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:05.057845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:05.058941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:05.094791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6ae63-207e-427a-9eaa-b254ab35f9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:05.095483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ae63-207e-427a-9eaa-b254ab35f9fe None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:05.095795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ae63-207e-427a-9eaa-b254ab35f9fe None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:05.096075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ae63-207e-427a-9eaa-b254ab35f9fe None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:05.096845 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6ae63-207e-427a-9eaa-b254ab35f9fe None admin] RBAC: Adding query filter params (name=reader) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:05.097153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6ae63-207e-427a-9eaa-b254ab35f9fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:05.098565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6ae63-207e-427a-9eaa-b254ab35f9fe None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:05.113243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 76/155] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:05.119679 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6ae63-207e-427a-9eaa-b254ab35f9fe None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:05.254017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feec6a8-47f4-4896-ab1d-8d9a542dd996 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:05.283761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6ae63-207e-427a-9eaa-b254ab35f9fe None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:05.341388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4237345-c3be-4fa2-a931-9b947e23f6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:05.342212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4237345-c3be-4fa2-a931-9b947e23f6ce None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:05.342504 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4237345-c3be-4fa2-a931-9b947e23f6ce None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:05.342786 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4237345-c3be-4fa2-a931-9b947e23f6ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:05.373611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4237345-c3be-4fa2-a931-9b947e23f6ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:05.386458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c1d5c-5a12-495e-88d1-b4d2847a3f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:05.387361 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c1d5c-5a12-495e-88d1-b4d2847a3f29 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:05.387620 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c1d5c-5a12-495e-88d1-b4d2847a3f29 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:05.387917 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c1d5c-5a12-495e-88d1-b4d2847a3f29 None admin] PATH_INFO: `/v3/users/80b704353f364eb798f11553af1fd8a5` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-f4237345-c3be-4fa2-a931-9b947e23f6ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:05.390090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:05.391092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:05.391945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:05.391945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:05.391945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:05.391945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:05.391945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:05.391945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:05.391945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:05.391945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:05.391945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:05.391945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:05.391945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:38:05.391945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:05.392523 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c1d5c-5a12-495e-88d1-b4d2847a3f29 None admin] RBAC: Authorizing `identity:get_user(user_id=80b704353f364eb798f11553af1fd8a5)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:05.392523 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c1d5c-5a12-495e-88d1-b4d2847a3f29 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:05.396104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 77/156] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:05 2022] GET /identity/v3/users/80b704353f364eb798f11553af1fd8a5 => generated 289 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:05.397395 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c06686-4bce-43b4-a952-9e2a8638f33b None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:05.397676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c06686-4bce-43b4-a952-9e2a8638f33b None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:05.398532 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c06686-4bce-43b4-a952-9e2a8638f33b None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:05.399381 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 80/157] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:38:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:05.404503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4237345-c3be-4fa2-a931-9b947e23f6ce None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:05.405587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 78/158] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:05.406730 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c06686-4bce-43b4-a952-9e2a8638f33b None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:05.464089 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:05.550527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c06686-4bce-43b4-a952-9e2a8638f33b None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:05.624074 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:05.624854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1437d6-786a-400b-b868-b7f695251993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:05.625000 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1437d6-786a-400b-b868-b7f695251993 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:05.625327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1437d6-786a-400b-b868-b7f695251993 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:05.625565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1437d6-786a-400b-b868-b7f695251993 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:05.626283 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1437d6-786a-400b-b868-b7f695251993 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:05.626584 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1437d6-786a-400b-b868-b7f695251993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:05.628004 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1437d6-786a-400b-b868-b7f695251993 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:05.641767 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4237345-c3be-4fa2-a931-9b947e23f6ce None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:05.646014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 79/159] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:05.647307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b0428-094a-4133-bc42-fbaca7e5ad8b None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:05.647588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b0428-094a-4133-bc42-fbaca7e5ad8b None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:05.656623 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b0428-094a-4133-bc42-fbaca7e5ad8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:05.687291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-f84b0428-094a-4133-bc42-fbaca7e5ad8b None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:05.688517 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f84b0428-094a-4133-bc42-fbaca7e5ad8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:05.714476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51365f2d-d44d-4ebe-a221-b0e8e066ed4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:05.715344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51365f2d-d44d-4ebe-a221-b0e8e066ed4e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:05.715627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51365f2d-d44d-4ebe-a221-b0e8e066ed4e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:05.715922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51365f2d-d44d-4ebe-a221-b0e8e066ed4e None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:05.718410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51365f2d-d44d-4ebe-a221-b0e8e066ed4e None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:05.719896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51365f2d-d44d-4ebe-a221-b0e8e066ed4e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:05.721911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 81/160] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:05 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:05.722373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51365f2d-d44d-4ebe-a221-b0e8e066ed4e None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:05.827822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 80/161] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:05.827822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b0428-094a-4133-bc42-fbaca7e5ad8b None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:05.873554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:05.904882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51365f2d-d44d-4ebe-a221-b0e8e066ed4e None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:05.942907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f01c5-0ac8-4ed7-ae5d-ac7bb9aa3c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:05.943982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f01c5-0ac8-4ed7-ae5d-ac7bb9aa3c09 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:05.944371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f01c5-0ac8-4ed7-ae5d-ac7bb9aa3c09 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:05.945272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f01c5-0ac8-4ed7-ae5d-ac7bb9aa3c09 None admin] PATH_INFO: `/v3/users/5ca745bbd8294b9eaae8eb567675a772` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:05.947294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f01c5-0ac8-4ed7-ae5d-ac7bb9aa3c09 None admin] RBAC: Authorizing `identity:get_user(user_id=5ca745bbd8294b9eaae8eb567675a772)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:05.948603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f01c5-0ac8-4ed7-ae5d-ac7bb9aa3c09 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:05.950552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 82/162] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:05 2022] GET /identity/v3/users/5ca745bbd8294b9eaae8eb567675a772 => generated 247 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:05.955734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294a9a7-0f0b-4332-889a-d7d8740a1279 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:05.955734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294a9a7-0f0b-4332-889a-d7d8740a1279 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:05.955734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294a9a7-0f0b-4332-889a-d7d8740a1279 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:05.955734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 83/163] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:05.955734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294a9a7-0f0b-4332-889a-d7d8740a1279 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:05.966621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b0428-094a-4133-bc42-fbaca7e5ad8b None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:05.988094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:06.017331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1feab-9c19-4517-9dec-c231dfe4ab7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:06.018474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1feab-9c19-4517-9dec-c231dfe4ab7d None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:06.018898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1feab-9c19-4517-9dec-c231dfe4ab7d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:06.019300 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1feab-9c19-4517-9dec-c231dfe4ab7d None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823/users/80b704353f364eb798f11553af1fd8a5/roles/1052f958732d4259a0658b40d1f9c193` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:06.025012 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1feab-9c19-4517-9dec-c231dfe4ab7d None admin] RBAC: Authorizing `identity:create_grant(project_id=32ebd8e81f314e8fa20cb9ba0042a823, user_id=80b704353f364eb798f11553af1fd8a5, role_id=1052f958732d4259a0658b40d1f9c193)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:06.026641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1feab-9c19-4517-9dec-c231dfe4ab7d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:06.053823 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 81/164] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:38:05 2022] PUT /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823/users/80b704353f364eb798f11553af1fd8a5/roles/1052f958732d4259a0658b40d1f9c193 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:38:06.055586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933f36f-4e1c-4736-8226-565600d3e0ad None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:06.056043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933f36f-4e1c-4736-8226-565600d3e0ad None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:06.056534 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933f36f-4e1c-4736-8226-565600d3e0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:06.070664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294a9a7-0f0b-4332-889a-d7d8740a1279 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:06.095047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d933f36f-4e1c-4736-8226-565600d3e0ad None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:06.096544 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d933f36f-4e1c-4736-8226-565600d3e0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:06.112844 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaaba99-81c4-4054-92e1-c2fcd8be4de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:06.114186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaaba99-81c4-4054-92e1-c2fcd8be4de5 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:06.114778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaaba99-81c4-4054-92e1-c2fcd8be4de5 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:06.115258 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaaba99-81c4-4054-92e1-c2fcd8be4de5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:06.129620 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaaba99-81c4-4054-92e1-c2fcd8be4de5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-7eaaba99-81c4-4054-92e1-c2fcd8be4de5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:06.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:06.140268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:06.141021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:06.141021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:06.141021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:06.141021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:06.141021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:06.141021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:06.141021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:06.141021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:06.141021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:06.141021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:06.141021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:06.141021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:06.143639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 84/165] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:38:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:06.144732 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaaba99-81c4-4054-92e1-c2fcd8be4de5 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:06.230860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 82/166] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:06.231764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933f36f-4e1c-4736-8226-565600d3e0ad None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:06.257198 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:06.264347 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:06.285756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaaba99-81c4-4054-92e1-c2fcd8be4de5 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:06.357596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9da67-3164-49df-84fd-d8652fb54d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:06.358649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9da67-3164-49df-84fd-d8652fb54d44 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:06.359141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9da67-3164-49df-84fd-d8652fb54d44 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:06.359622 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9da67-3164-49df-84fd-d8652fb54d44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:06.402750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933f36f-4e1c-4736-8226-565600d3e0ad None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:06.462586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b9da67-3164-49df-84fd-d8652fb54d44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:06.464293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b9da67-3164-49df-84fd-d8652fb54d44 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:06.470443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad454fc-5708-4569-9efc-ae3843a904b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:06.471790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad454fc-5708-4569-9efc-ae3843a904b8 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:06.472279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad454fc-5708-4569-9efc-ae3843a904b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:06.472832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad454fc-5708-4569-9efc-ae3843a904b8 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:06.473804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad454fc-5708-4569-9efc-ae3843a904b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:06.474337 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad454fc-5708-4569-9efc-ae3843a904b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:06.476684 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad454fc-5708-4569-9efc-ae3843a904b8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:06.490651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 83/167] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:06.491949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad454fc-5708-4569-9efc-ae3843a904b8 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:06.590810 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:06.610676 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 85/168] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:06.611651 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9da67-3164-49df-84fd-d8652fb54d44 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:06.660651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:06.687003 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad454fc-5708-4569-9efc-ae3843a904b8 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:06.789151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9c75f-3d3b-40a0-9326-51fc56ac84de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:06.789993 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9c75f-3d3b-40a0-9326-51fc56ac84de None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:06.790279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9c75f-3d3b-40a0-9326-51fc56ac84de None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:06.790541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9c75f-3d3b-40a0-9326-51fc56ac84de None admin] PATH_INFO: `/v3/roles/member` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:06.807574 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9c75f-3d3b-40a0-9326-51fc56ac84de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-daa9c75f-3d3b-40a0-9326-51fc56ac84de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:06.815376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:06.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:06.817143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:06.817143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:06.817143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:06.817143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:06.817143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:06.817143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:06.817143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:06.817143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:06.817143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:06.817143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:06.817143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:06.817143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:06.817143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 84/169] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:06.817775 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9c75f-3d3b-40a0-9326-51fc56ac84de None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:06.853887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9da67-3164-49df-84fd-d8652fb54d44 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:06.956575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cf0e5-8d05-4546-a55c-5eb781ec7016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:06.957402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cf0e5-8d05-4546-a55c-5eb781ec7016 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:06.957690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cf0e5-8d05-4546-a55c-5eb781ec7016 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:06.957971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cf0e5-8d05-4546-a55c-5eb781ec7016 None admin] PATH_INFO: `/v3/users/805b77c8bb67450db935957c46dc8b9b` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:06.959345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cf0e5-8d05-4546-a55c-5eb781ec7016 None admin] RBAC: Authorizing `identity:get_user(user_id=805b77c8bb67450db935957c46dc8b9b)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:06.961715 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cf0e5-8d05-4546-a55c-5eb781ec7016 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:06.963527 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 86/170] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:06 2022] GET /identity/v3/users/805b77c8bb67450db935957c46dc8b9b => generated 275 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:06.966767 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cf0e5-8d05-4546-a55c-5eb781ec7016 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:07.030320 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9c75f-3d3b-40a0-9326-51fc56ac84de None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:07.089305 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:07.102079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d61f8-5cdb-4911-850f-6d3124fcbf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:07.102950 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d61f8-5cdb-4911-850f-6d3124fcbf7c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:07.103229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d61f8-5cdb-4911-850f-6d3124fcbf7c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:07.103513 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d61f8-5cdb-4911-850f-6d3124fcbf7c None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:07.108447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d61f8-5cdb-4911-850f-6d3124fcbf7c None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:07.109918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d61f8-5cdb-4911-850f-6d3124fcbf7c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:07.111664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 85/171] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:06 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:07.112451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d61f8-5cdb-4911-850f-6d3124fcbf7c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:07.126872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cf0e5-8d05-4546-a55c-5eb781ec7016 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:07.136108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:07.169885 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98536261-649a-48be-861e-eebd635cc3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:07.170604 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98536261-649a-48be-861e-eebd635cc3b1 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:07.170887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98536261-649a-48be-861e-eebd635cc3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:07.171211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98536261-649a-48be-861e-eebd635cc3b1 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:07.172006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98536261-649a-48be-861e-eebd635cc3b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:07.172311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98536261-649a-48be-861e-eebd635cc3b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:07.173731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98536261-649a-48be-861e-eebd635cc3b1 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:07.183386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 87/172] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:07.185069 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973eb0e-dffc-4b6f-b157-c98145c89cf8 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:07.185359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973eb0e-dffc-4b6f-b157-c98145c89cf8 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:07.185638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973eb0e-dffc-4b6f-b157-c98145c89cf8 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:07.187048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 88/173] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:07.187938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973eb0e-dffc-4b6f-b157-c98145c89cf8 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:07.295203 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d61f8-5cdb-4911-850f-6d3124fcbf7c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:07.307371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973eb0e-dffc-4b6f-b157-c98145c89cf8 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:07.328517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:07.346352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b252d15-aad1-424e-bb2d-1f3fd7a140d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:07.348258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b252d15-aad1-424e-bb2d-1f3fd7a140d0 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:07.348525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b252d15-aad1-424e-bb2d-1f3fd7a140d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:07.349165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b252d15-aad1-424e-bb2d-1f3fd7a140d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:07.356195 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d90475-4ceb-4512-973f-8434028bef82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:07.357305 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d90475-4ceb-4512-973f-8434028bef82 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:07.357793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d90475-4ceb-4512-973f-8434028bef82 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:07.358272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d90475-4ceb-4512-973f-8434028bef82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:07.359412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d90475-4ceb-4512-973f-8434028bef82 None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823, user.id=5ca745bbd8294b9eaae8eb567675a772) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:07.359997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d90475-4ceb-4512-973f-8434028bef82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:07.361809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d90475-4ceb-4512-973f-8434028bef82 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:07.370047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 89/174] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:07 2022] GET /identity/v3/role_assignments?user.id=5ca745bbd8294b9eaae8eb567675a772&scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:07.371386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d90475-4ceb-4512-973f-8434028bef82 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:07.459856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b252d15-aad1-424e-bb2d-1f3fd7a140d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:07.462671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b252d15-aad1-424e-bb2d-1f3fd7a140d0 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:07.478631 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:07.623339 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 86/175] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:07.625250 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f244b-026f-4006-b3f0-96598a7d17d3 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:07.625753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f244b-026f-4006-b3f0-96598a7d17d3 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:07.626245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f244b-026f-4006-b3f0-96598a7d17d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:07.635224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d90475-4ceb-4512-973f-8434028bef82 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:07.646439 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-1d5f244b-026f-4006-b3f0-96598a7d17d3 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:07.649011 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1d5f244b-026f-4006-b3f0-96598a7d17d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:07.684852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b20e57-4433-4d34-9bfc-faff988a9c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:07.685750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b20e57-4433-4d34-9bfc-faff988a9c03 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:07.686043 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b20e57-4433-4d34-9bfc-faff988a9c03 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:07.686328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b20e57-4433-4d34-9bfc-faff988a9c03 None admin] PATH_INFO: `/v3/users/805b77c8bb67450db935957c46dc8b9b` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:07.687570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b20e57-4433-4d34-9bfc-faff988a9c03 None admin] RBAC: Authorizing `identity:get_user(user_id=805b77c8bb67450db935957c46dc8b9b)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:07.689073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b20e57-4433-4d34-9bfc-faff988a9c03 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:07.696543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 90/176] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:07 2022] GET /identity/v3/users/805b77c8bb67450db935957c46dc8b9b => generated 275 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:07.696543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5412a-6b2d-49d7-a7d3-9d5aabcbd203 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:07.696543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5412a-6b2d-49d7-a7d3-9d5aabcbd203 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:07.696543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5412a-6b2d-49d7-a7d3-9d5aabcbd203 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:07.696543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 91/177] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:07.696942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1c27e-50ac-48e8-a6dd-de06d5e09496 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:07.697219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1c27e-50ac-48e8-a6dd-de06d5e09496 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:07.697495 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1c27e-50ac-48e8-a6dd-de06d5e09496 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:07.698917 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 92/178] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:07.699470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1c27e-50ac-48e8-a6dd-de06d5e09496 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:07.734438 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 87/179] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:07.737008 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ebd20-5a4d-438b-8d20-d2edc9e3ff98 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:07.737491 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ebd20-5a4d-438b-8d20-d2edc9e3ff98 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:07.737945 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ebd20-5a4d-438b-8d20-d2edc9e3ff98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:07.769236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-157ebd20-5a4d-438b-8d20-d2edc9e3ff98 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:07.770836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-157ebd20-5a4d-438b-8d20-d2edc9e3ff98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:07.804644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:07.833841 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:07.881590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1c27e-50ac-48e8-a6dd-de06d5e09496 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:07.882385 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 88/180] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:07.883544 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ebd20-5a4d-438b-8d20-d2edc9e3ff98 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:07.940452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdf221-cf65-4ca2-afda-35495f4bb64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:07.941725 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdf221-cf65-4ca2-afda-35495f4bb64c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:07.942216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdf221-cf65-4ca2-afda-35495f4bb64c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:07.942694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdf221-cf65-4ca2-afda-35495f4bb64c None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:07.944186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdf221-cf65-4ca2-afda-35495f4bb64c None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:07.946510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdf221-cf65-4ca2-afda-35495f4bb64c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:07.948523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 93/181] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:07 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:07.950032 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea9220-2f35-46d3-9da5-c3450722517b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:07.950534 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea9220-2f35-46d3-9da5-c3450722517b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:07.951015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea9220-2f35-46d3-9da5-c3450722517b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:07.971273 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-ceea9220-2f35-46d3-9da5-c3450722517b None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:07.972852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ceea9220-2f35-46d3-9da5-c3450722517b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:08.040873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ebd20-5a4d-438b-8d20-d2edc9e3ff98 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:08.061796 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 94/182] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:08.068184 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea9220-2f35-46d3-9da5-c3450722517b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:08.111797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8ea30-1ecb-4f75-a8cb-ad201643a90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:08.112500 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8ea30-1ecb-4f75-a8cb-ad201643a90f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:08.112789 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8ea30-1ecb-4f75-a8cb-ad201643a90f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:08.113051 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8ea30-1ecb-4f75-a8cb-ad201643a90f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:08.162540 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:08.170791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:08.207955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8ea30-1ecb-4f75-a8cb-ad201643a90f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:08.209708 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8ea30-1ecb-4f75-a8cb-ad201643a90f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:08.227132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea9220-2f35-46d3-9da5-c3450722517b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:08.267454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea289f-163a-4883-9575-ff243ababe56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:08.268297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea289f-163a-4883-9575-ff243ababe56 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:08.268582 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea289f-163a-4883-9575-ff243ababe56 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:08.268857 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea289f-163a-4883-9575-ff243ababe56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:08.285226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea289f-163a-4883-9575-ff243ababe56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-edea289f-163a-4883-9575-ff243ababe56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:08.295298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:08.296278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:08.297022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:08.297022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:08.297022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:08.297022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:08.297022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:08.297022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:08.297022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:08.297022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:08.297022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:08.297022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:08.297022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:38:08.297022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:08.297591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 95/183] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:38:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:08.298497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea289f-163a-4883-9575-ff243ababe56 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:08.330367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 89/184] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:08.331250 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8ea30-1ecb-4f75-a8cb-ad201643a90f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:08.420134 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:08.430048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea289f-163a-4883-9575-ff243ababe56 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:08.439953 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:08.457693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8ea30-1ecb-4f75-a8cb-ad201643a90f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:08.471400 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300f969-fb4a-4a7d-80cd-50dcb4418d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:08.472258 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300f969-fb4a-4a7d-80cd-50dcb4418d64 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:08.472538 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300f969-fb4a-4a7d-80cd-50dcb4418d64 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:08.472844 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300f969-fb4a-4a7d-80cd-50dcb4418d64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:08.485097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300f969-fb4a-4a7d-80cd-50dcb4418d64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-f300f969-fb4a-4a7d-80cd-50dcb4418d64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:08.493615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:08.494519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:08.500595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:08.500595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:08.500595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:08.500595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:08.500595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:08.500595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:08.500595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:08.500595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:08.500595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:08.500595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:08.500595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:08.500595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:08.500595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 96/185] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:38:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:08.500595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300f969-fb4a-4a7d-80cd-50dcb4418d64 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:08.509355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b45d8-c1a5-4f49-b507-8bf62f3e96f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:08.509355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b45d8-c1a5-4f49-b507-8bf62f3e96f7 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:08.509355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b45d8-c1a5-4f49-b507-8bf62f3e96f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:08.509355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b45d8-c1a5-4f49-b507-8bf62f3e96f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:08.510054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b45d8-c1a5-4f49-b507-8bf62f3e96f7 None admin] RBAC: Adding query filter params (role.id=04597f92ac654fb4ac68dbb783d3dc31, scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823, user.id=805b77c8bb67450db935957c46dc8b9b) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:08.510054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b45d8-c1a5-4f49-b507-8bf62f3e96f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:08.510345 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b45d8-c1a5-4f49-b507-8bf62f3e96f7 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:08.524527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 90/186] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:08 2022] GET /identity/v3/role_assignments?user.id=805b77c8bb67450db935957c46dc8b9b&scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823&role.id=04597f92ac654fb4ac68dbb783d3dc31 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:08.524527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b45d8-c1a5-4f49-b507-8bf62f3e96f7 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:08.639937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300f969-fb4a-4a7d-80cd-50dcb4418d64 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:08.667193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b45d8-c1a5-4f49-b507-8bf62f3e96f7 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:08.687408 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8406f-be30-43d3-b086-ffee3120b45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:08.688169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8406f-be30-43d3-b086-ffee3120b45c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:08.688451 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8406f-be30-43d3-b086-ffee3120b45c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:08.688726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8406f-be30-43d3-b086-ffee3120b45c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:08.689991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8406f-be30-43d3-b086-ffee3120b45c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:08.691685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8406f-be30-43d3-b086-ffee3120b45c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:08.694619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 97/187] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:08.696406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8406f-be30-43d3-b086-ffee3120b45c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:08.715619 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb7ff8-57d7-4549-97dc-0244bc31aeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:08.716677 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb7ff8-57d7-4549-97dc-0244bc31aeda None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:08.716960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb7ff8-57d7-4549-97dc-0244bc31aeda None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:08.717402 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb7ff8-57d7-4549-97dc-0244bc31aeda None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:08.718461 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb7ff8-57d7-4549-97dc-0244bc31aeda None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:08.719299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb7ff8-57d7-4549-97dc-0244bc31aeda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:08.721043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb7ff8-57d7-4549-97dc-0244bc31aeda None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:08.723611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:08.740078 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 91/188] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:08.741794 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb7ff8-57d7-4549-97dc-0244bc31aeda None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:08.754914 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:08.855155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8406f-be30-43d3-b086-ffee3120b45c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:08.886116 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb7ff8-57d7-4549-97dc-0244bc31aeda None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:08.893566 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f6bf1-bdb2-4d0c-b9fb-ff89494fe9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:08.894441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f6bf1-bdb2-4d0c-b9fb-ff89494fe9f3 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:08.894726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f6bf1-bdb2-4d0c-b9fb-ff89494fe9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:08.895002 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f6bf1-bdb2-4d0c-b9fb-ff89494fe9f3 None admin] PATH_INFO: `/v3/projects/fee9d514657246fe80d694402f1870b0` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:08.896246 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f6bf1-bdb2-4d0c-b9fb-ff89494fe9f3 None admin] RBAC: Authorizing `identity:get_project(project_id=fee9d514657246fe80d694402f1870b0)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:08.897703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f6bf1-bdb2-4d0c-b9fb-ff89494fe9f3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:08.900314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 98/189] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:08 2022] GET /identity/v3/projects/fee9d514657246fe80d694402f1870b0 => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:08.901288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f6bf1-bdb2-4d0c-b9fb-ff89494fe9f3 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:08.925279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83671d-5cb4-4523-9c86-d23e7a1e6a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:08.925953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83671d-5cb4-4523-9c86-d23e7a1e6a5b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:08.926237 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83671d-5cb4-4523-9c86-d23e7a1e6a5b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:08.926516 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83671d-5cb4-4523-9c86-d23e7a1e6a5b None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:08.927271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83671d-5cb4-4523-9c86-d23e7a1e6a5b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:08.927578 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83671d-5cb4-4523-9c86-d23e7a1e6a5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:08.929002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83671d-5cb4-4523-9c86-d23e7a1e6a5b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:08.938025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 92/190] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:08.938799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83671d-5cb4-4523-9c86-d23e7a1e6a5b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:09.002797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:09.021513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f6bf1-bdb2-4d0c-b9fb-ff89494fe9f3 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:09.024581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:09.048046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83671d-5cb4-4523-9c86-d23e7a1e6a5b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:09.062472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dbe45-173e-44f4-8b43-30b1ddbeea8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:09.063157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dbe45-173e-44f4-8b43-30b1ddbeea8a None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:09.063443 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dbe45-173e-44f4-8b43-30b1ddbeea8a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:09.063752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dbe45-173e-44f4-8b43-30b1ddbeea8a None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:09.064484 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dbe45-173e-44f4-8b43-30b1ddbeea8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:09.065940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dbe45-173e-44f4-8b43-30b1ddbeea8a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:09.086862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb1134-c112-412b-9f90-eff27dbb5d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:09.088170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb1134-c112-412b-9f90-eff27dbb5d2a None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:09.088490 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb1134-c112-412b-9f90-eff27dbb5d2a None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:09.088770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb1134-c112-412b-9f90-eff27dbb5d2a None admin] PATH_INFO: `/v3/users/7805fa8ea7214ce2b5c934735e50d7ca` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:09.098422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 99/191] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:38:08 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:38:09.099878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8996d6-1210-4418-a8ef-c86aafead544 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:09.100174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8996d6-1210-4418-a8ef-c86aafead544 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:09.100500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8996d6-1210-4418-a8ef-c86aafead544 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:09.103097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 100/192] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:09.105302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb1134-c112-412b-9f90-eff27dbb5d2a None admin] RBAC: Authorizing `identity:get_user(user_id=7805fa8ea7214ce2b5c934735e50d7ca)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:09.106891 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb1134-c112-412b-9f90-eff27dbb5d2a None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:09.109601 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8996d6-1210-4418-a8ef-c86aafead544 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:09.111941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 93/193] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:08 2022] GET /identity/v3/users/7805fa8ea7214ce2b5c934735e50d7ca => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:09.114593 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb1134-c112-412b-9f90-eff27dbb5d2a None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:09.245010 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8996d6-1210-4418-a8ef-c86aafead544 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:09.246161 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb1134-c112-412b-9f90-eff27dbb5d2a None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:09.267937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:09.292159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54c995-a433-4070-8b5c-775ec7f27dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:09.292530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54c995-a433-4070-8b5c-775ec7f27dd6 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:09.292823 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54c995-a433-4070-8b5c-775ec7f27dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:09.293099 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54c995-a433-4070-8b5c-775ec7f27dd6 None admin] PATH_INFO: `/v3/users/7805fa8ea7214ce2b5c934735e50d7ca` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:09.294432 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54c995-a433-4070-8b5c-775ec7f27dd6 None admin] RBAC: Authorizing `identity:get_user(user_id=7805fa8ea7214ce2b5c934735e50d7ca)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:09.296021 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54c995-a433-4070-8b5c-775ec7f27dd6 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:09.302997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 94/194] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:09 2022] GET /identity/v3/users/7805fa8ea7214ce2b5c934735e50d7ca => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:09.303173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc6c29-a77a-4006-b95a-8c918f0b1141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:09.303173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc6c29-a77a-4006-b95a-8c918f0b1141 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:09.303173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc6c29-a77a-4006-b95a-8c918f0b1141 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:09.303393 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210997af-69fe-4dc4-a233-277c80a313be None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:09.303459 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc6c29-a77a-4006-b95a-8c918f0b1141 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:09.303524 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210997af-69fe-4dc4-a233-277c80a313be None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:09.303524 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210997af-69fe-4dc4-a233-277c80a313be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:09.303649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc6c29-a77a-4006-b95a-8c918f0b1141 None admin] RBAC: Adding query filter params (role.id=22f9190503114c4aba5e898e7a460546, scope.project.id=fee9d514657246fe80d694402f1870b0, user.id=805b77c8bb67450db935957c46dc8b9b) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:09.303649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc6c29-a77a-4006-b95a-8c918f0b1141 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:09.303649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc6c29-a77a-4006-b95a-8c918f0b1141 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:09.310894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 101/195] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:09 2022] GET /identity/v3/role_assignments?user.id=805b77c8bb67450db935957c46dc8b9b&scope.project.id=fee9d514657246fe80d694402f1870b0&role.id=22f9190503114c4aba5e898e7a460546 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:09.321517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc6c29-a77a-4006-b95a-8c918f0b1141 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:09.327397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-210997af-69fe-4dc4-a233-277c80a313be None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:09.328762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-210997af-69fe-4dc4-a233-277c80a313be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:09.358611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:09.428331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 95/196] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:09.429875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210997af-69fe-4dc4-a233-277c80a313be None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:09.489454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc6c29-a77a-4006-b95a-8c918f0b1141 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:09.532959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1038ed-40f1-40c5-9de6-2886d4a4492b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:09.533527 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1038ed-40f1-40c5-9de6-2886d4a4492b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:09.533815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1038ed-40f1-40c5-9de6-2886d4a4492b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:09.534096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1038ed-40f1-40c5-9de6-2886d4a4492b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:09.584824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210997af-69fe-4dc4-a233-277c80a313be None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:09.608418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:09.633126 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1038ed-40f1-40c5-9de6-2886d4a4492b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:09.636050 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1038ed-40f1-40c5-9de6-2886d4a4492b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:09.636906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415d7a5-ef38-4764-86ae-5280bfb95a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:09.637803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415d7a5-ef38-4764-86ae-5280bfb95a8f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:09.638549 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415d7a5-ef38-4764-86ae-5280bfb95a8f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:09.639061 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415d7a5-ef38-4764-86ae-5280bfb95a8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:09.675400 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:09.748259 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415d7a5-ef38-4764-86ae-5280bfb95a8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:09.750447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415d7a5-ef38-4764-86ae-5280bfb95a8f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:09.779079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 102/197] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:09.783948 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1038ed-40f1-40c5-9de6-2886d4a4492b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:09.881771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 96/198] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:09.883015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415d7a5-ef38-4764-86ae-5280bfb95a8f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:09.903415 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:09.927488 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1038ed-40f1-40c5-9de6-2886d4a4492b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:09.955125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:09.968112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570d7bd-b374-43c9-aa37-f9e4e42aa8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:09.968922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570d7bd-b374-43c9-aa37-f9e4e42aa8bf None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:09.969204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570d7bd-b374-43c9-aa37-f9e4e42aa8bf None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:09.969483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570d7bd-b374-43c9-aa37-f9e4e42aa8bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:09.988961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570d7bd-b374-43c9-aa37-f9e4e42aa8bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-e570d7bd-b374-43c9-aa37-f9e4e42aa8bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:10.000562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:10.001486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:10.002238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:10.002238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:10.002238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:10.002238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:10.002238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:10.002238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:10.002238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:10.002238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:10.002238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:10.002238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:10.002238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:10.002238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:10.004965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 103/199] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:10.010240 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570d7bd-b374-43c9-aa37-f9e4e42aa8bf None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:10.040724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415d7a5-ef38-4764-86ae-5280bfb95a8f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:10.090261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787f4df-2b42-4a95-8604-7d20b50fe70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:10.091453 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787f4df-2b42-4a95-8604-7d20b50fe70e None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:10.091970 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787f4df-2b42-4a95-8604-7d20b50fe70e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:10.092426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787f4df-2b42-4a95-8604-7d20b50fe70e None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:10.093917 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787f4df-2b42-4a95-8604-7d20b50fe70e None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:10.096072 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787f4df-2b42-4a95-8604-7d20b50fe70e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:10.098404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 97/200] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:09 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:10.099742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787f4df-2b42-4a95-8604-7d20b50fe70e None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:10.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570d7bd-b374-43c9-aa37-f9e4e42aa8bf None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:10.179017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e6af5-39f7-405c-b369-52be7883fa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:10.180383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e6af5-39f7-405c-b369-52be7883fa57 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:10.181106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e6af5-39f7-405c-b369-52be7883fa57 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:10.181583 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e6af5-39f7-405c-b369-52be7883fa57 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:10.183875 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e6af5-39f7-405c-b369-52be7883fa57 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:10.185826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e6af5-39f7-405c-b369-52be7883fa57 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:10.188595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 104/201] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:10 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:10.189546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e6af5-39f7-405c-b369-52be7883fa57 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:10.205922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:10.228155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787f4df-2b42-4a95-8604-7d20b50fe70e None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:10.255735 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:10.274893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15c0d1-8950-407c-83f8-7473fa9950dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:10.275678 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15c0d1-8950-407c-83f8-7473fa9950dc None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:10.276055 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15c0d1-8950-407c-83f8-7473fa9950dc None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:10.276584 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15c0d1-8950-407c-83f8-7473fa9950dc None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:10.277366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15c0d1-8950-407c-83f8-7473fa9950dc None admin] RBAC: Adding query filter params (name=reader) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:10.277673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15c0d1-8950-407c-83f8-7473fa9950dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:10.279774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15c0d1-8950-407c-83f8-7473fa9950dc None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:10.292880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 98/202] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:10.295904 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15c0d1-8950-407c-83f8-7473fa9950dc None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:10.320165 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e6af5-39f7-405c-b369-52be7883fa57 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:10.364045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff481c20-a650-4b32-8621-4fd327a45a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:10.364722 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff481c20-a650-4b32-8621-4fd327a45a46 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:10.365004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff481c20-a650-4b32-8621-4fd327a45a46 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:10.365281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff481c20-a650-4b32-8621-4fd327a45a46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:10.366126 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff481c20-a650-4b32-8621-4fd327a45a46 None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=3fa3d431d9cc468aafe7b895f047ff94, user.id=7805fa8ea7214ce2b5c934735e50d7ca) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:10.366440 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff481c20-a650-4b32-8621-4fd327a45a46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:10.368479 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff481c20-a650-4b32-8621-4fd327a45a46 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:10.376061 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 105/203] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:10 2022] GET /identity/v3/role_assignments?user.id=7805fa8ea7214ce2b5c934735e50d7ca&scope.project.id=3fa3d431d9cc468aafe7b895f047ff94&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:10.376968 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff481c20-a650-4b32-8621-4fd327a45a46 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:10.517029 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15c0d1-8950-407c-83f8-7473fa9950dc None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:10.534501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff481c20-a650-4b32-8621-4fd327a45a46 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:10.571089 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e9607-13aa-4437-b303-ce1637bcf02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:10.573793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e9607-13aa-4437-b303-ce1637bcf02e None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:10.574140 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e9607-13aa-4437-b303-ce1637bcf02e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:10.574425 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e9607-13aa-4437-b303-ce1637bcf02e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:10.575324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e9607-13aa-4437-b303-ce1637bcf02e None admin] RBAC: Adding query filter params (role.id=04597f92ac654fb4ac68dbb783d3dc31, scope.project.id=3fa3d431d9cc468aafe7b895f047ff94, user.id=7805fa8ea7214ce2b5c934735e50d7ca) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:10.575911 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e9607-13aa-4437-b303-ce1637bcf02e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:10.578271 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:10.579573 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e9607-13aa-4437-b303-ce1637bcf02e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:10.592222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 99/204] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:10 2022] GET /identity/v3/role_assignments?user.id=7805fa8ea7214ce2b5c934735e50d7ca&scope.project.id=3fa3d431d9cc468aafe7b895f047ff94&role.id=04597f92ac654fb4ac68dbb783d3dc31 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:10.609357 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebedc57-9afa-4f67-a5a9-ae32cfcbabc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:10.610279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebedc57-9afa-4f67-a5a9-ae32cfcbabc8 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:10.610557 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebedc57-9afa-4f67-a5a9-ae32cfcbabc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:10.610832 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebedc57-9afa-4f67-a5a9-ae32cfcbabc8 None admin] PATH_INFO: `/v3/users/80b704353f364eb798f11553af1fd8a5` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:10.612391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebedc57-9afa-4f67-a5a9-ae32cfcbabc8 None admin] RBAC: Authorizing `identity:get_user(user_id=80b704353f364eb798f11553af1fd8a5)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:10.613977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebedc57-9afa-4f67-a5a9-ae32cfcbabc8 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:10.616225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 106/205] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:10 2022] GET /identity/v3/users/80b704353f364eb798f11553af1fd8a5 => generated 289 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:10.622068 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e9607-13aa-4437-b303-ce1637bcf02e None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:10.644197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:10.771423 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e9607-13aa-4437-b303-ce1637bcf02e None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:10.810987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341c9f7-b282-485c-a8e1-18668e4251d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:10.811667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341c9f7-b282-485c-a8e1-18668e4251d8 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:10.811990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341c9f7-b282-485c-a8e1-18668e4251d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:10.812277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341c9f7-b282-485c-a8e1-18668e4251d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:10.890512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341c9f7-b282-485c-a8e1-18668e4251d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:10.891946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341c9f7-b282-485c-a8e1-18668e4251d8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:10.903644 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:11.002456 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 100/206] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:11.008873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebedc57-9afa-4f67-a5a9-ae32cfcbabc8 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:11.162626 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebedc57-9afa-4f67-a5a9-ae32cfcbabc8 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:11.208299 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9c502-72f0-4c55-a01d-a5a2754fb34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:11.209245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9c502-72f0-4c55-a01d-a5a2754fb34a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:11.209541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9c502-72f0-4c55-a01d-a5a2754fb34a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:11.209825 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9c502-72f0-4c55-a01d-a5a2754fb34a None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:11.211245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9c502-72f0-4c55-a01d-a5a2754fb34a None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:11.212957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9c502-72f0-4c55-a01d-a5a2754fb34a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:11.214711 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 107/207] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:11 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:11.220765 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341c9f7-b282-485c-a8e1-18668e4251d8 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:11.363751 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341c9f7-b282-485c-a8e1-18668e4251d8 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:11.381983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:11.418792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aeaa23-7ab5-4e54-914b-486f0fde62c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:11.419384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aeaa23-7ab5-4e54-914b-486f0fde62c3 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:11.419664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aeaa23-7ab5-4e54-914b-486f0fde62c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:11.419966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aeaa23-7ab5-4e54-914b-486f0fde62c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:11.420819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aeaa23-7ab5-4e54-914b-486f0fde62c3 None admin] RBAC: Adding query filter params (role.id=1052f958732d4259a0658b40d1f9c193, scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823, user.id=80b704353f364eb798f11553af1fd8a5) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:11.421130 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aeaa23-7ab5-4e54-914b-486f0fde62c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:11.422597 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aeaa23-7ab5-4e54-914b-486f0fde62c3 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:11.430668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 101/208] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:11 2022] GET /identity/v3/role_assignments?user.id=80b704353f364eb798f11553af1fd8a5&scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823&role.id=1052f958732d4259a0658b40d1f9c193 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:11.789883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26ea84-3faf-469e-ae2d-03003f09da0f None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:11.790217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26ea84-3faf-469e-ae2d-03003f09da0f None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:11.790394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26ea84-3faf-469e-ae2d-03003f09da0f None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:11.791991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 108/209] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:11.797586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbd465-698f-455a-a462-2991f0259bb9 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:11.798026 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbd465-698f-455a-a462-2991f0259bb9 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:11.798194 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbd465-698f-455a-a462-2991f0259bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:11.818531 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-f1dbd465-698f-455a-a462-2991f0259bb9 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:11.820305 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f1dbd465-698f-455a-a462-2991f0259bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:11.843389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de01ca1-fc17-4424-93ad-5ef06d3553ed None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:11.843697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de01ca1-fc17-4424-93ad-5ef06d3553ed None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:11.843984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de01ca1-fc17-4424-93ad-5ef06d3553ed None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:11.845526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 109/210] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:38:11.851692 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3778d47-a085-444e-a45a-253adf187ea5 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:11.852123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3778d47-a085-444e-a45a-253adf187ea5 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:11.852373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3778d47-a085-444e-a45a-253adf187ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:11.880824 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-b3778d47-a085-444e-a45a-253adf187ea5 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:11.884259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b3778d47-a085-444e-a45a-253adf187ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:11.943767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 102/211] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:38:11.959425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:11.982541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbd465-698f-455a-a462-2991f0259bb9 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:11.987542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 110/212] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:12.016264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3778d47-a085-444e-a45a-253adf187ea5 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:12.122220 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbd465-698f-455a-a462-2991f0259bb9 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:12.125255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:12.150193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3778d47-a085-444e-a45a-253adf187ea5 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:12.165131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8427389-b333-44be-abee-a6b8fc899f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:12.165958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8427389-b333-44be-abee-a6b8fc899f96 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:12.166226 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8427389-b333-44be-abee-a6b8fc899f96 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:12.166499 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8427389-b333-44be-abee-a6b8fc899f96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:12.167706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8427389-b333-44be-abee-a6b8fc899f96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:12.169315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8427389-b333-44be-abee-a6b8fc899f96 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:12.175764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 103/213] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:12.176906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8427389-b333-44be-abee-a6b8fc899f96 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:12.199683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6807d3c-9a9f-4d28-9960-a0ad76169d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:12.201024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6807d3c-9a9f-4d28-9960-a0ad76169d73 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:12.201559 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6807d3c-9a9f-4d28-9960-a0ad76169d73 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:12.202059 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6807d3c-9a9f-4d28-9960-a0ad76169d73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:12.216016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6807d3c-9a9f-4d28-9960-a0ad76169d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-e6807d3c-9a9f-4d28-9960-a0ad76169d73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:12.224655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:12.225574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:12.226420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:12.226420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:12.226420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:12.226420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:12.226420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:12.226420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:12.226420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:12.226420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:12.226420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:12.226420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:12.226420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:12.226420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:12.234073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 111/214] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:12.235342 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6807d3c-9a9f-4d28-9960-a0ad76169d73 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:12.283771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:12.296199 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8427389-b333-44be-abee-a6b8fc899f96 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:12.337455 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2af983-d456-4730-b33e-90bd470bad2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:12.338146 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2af983-d456-4730-b33e-90bd470bad2b None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:12.338436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2af983-d456-4730-b33e-90bd470bad2b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:12.338718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2af983-d456-4730-b33e-90bd470bad2b None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:12.339755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2af983-d456-4730-b33e-90bd470bad2b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:12.342030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2af983-d456-4730-b33e-90bd470bad2b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:12.350842 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6807d3c-9a9f-4d28-9960-a0ad76169d73 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:12.373080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 104/215] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:38:12 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:38:12.413118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3337488-7810-4a48-8ac2-acdc2f104d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:12.415190 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3337488-7810-4a48-8ac2-acdc2f104d52 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:12.415743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3337488-7810-4a48-8ac2-acdc2f104d52 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:12.416278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3337488-7810-4a48-8ac2-acdc2f104d52 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:12.417271 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3337488-7810-4a48-8ac2-acdc2f104d52 None admin] RBAC: Adding query filter params (name=member) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:12.417786 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3337488-7810-4a48-8ac2-acdc2f104d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:12.419479 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3337488-7810-4a48-8ac2-acdc2f104d52 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:12.431190 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 112/216] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:12.437585 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2af983-d456-4730-b33e-90bd470bad2b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:12.478913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:12.578941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2af983-d456-4730-b33e-90bd470bad2b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:12.618564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70628753-6a78-4114-96f9-f83ae9ecdee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:12.619467 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70628753-6a78-4114-96f9-f83ae9ecdee6 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:12.619782 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70628753-6a78-4114-96f9-f83ae9ecdee6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:12.620088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70628753-6a78-4114-96f9-f83ae9ecdee6 None admin] PATH_INFO: `/v3/users/9f1087e8c7e14a5d87c81fdc977088f1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:12.636555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70628753-6a78-4114-96f9-f83ae9ecdee6 None admin] RBAC: Authorizing `identity:get_user(user_id=9f1087e8c7e14a5d87c81fdc977088f1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:12.637982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70628753-6a78-4114-96f9-f83ae9ecdee6 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:12.640242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 105/217] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:12 2022] GET /identity/v3/users/9f1087e8c7e14a5d87c81fdc977088f1 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:12.645724 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3337488-7810-4a48-8ac2-acdc2f104d52 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:12.779181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3337488-7810-4a48-8ac2-acdc2f104d52 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:12.789816 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:12.826592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b410e-c7f8-4bd0-b475-d3cbb4dba226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:12.827667 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b410e-c7f8-4bd0-b475-d3cbb4dba226 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:12.828244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b410e-c7f8-4bd0-b475-d3cbb4dba226 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:12.828727 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b410e-c7f8-4bd0-b475-d3cbb4dba226 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:12.926437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b410e-c7f8-4bd0-b475-d3cbb4dba226 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:12.927864 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b410e-c7f8-4bd0-b475-d3cbb4dba226 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:12.981896 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476a43c-f6d2-4b22-acdc-e95f92d0290d None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:12.982859 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476a43c-f6d2-4b22-acdc-e95f92d0290d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:12.983348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476a43c-f6d2-4b22-acdc-e95f92d0290d None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:12.985019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 106/218] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:38:12.992306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608996a0-bb8f-4846-8c33-281ac51b400c None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:12.992918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608996a0-bb8f-4846-8c33-281ac51b400c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:12.993484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608996a0-bb8f-4846-8c33-281ac51b400c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:13.030211 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-608996a0-bb8f-4846-8c33-281ac51b400c None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:13.031793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-608996a0-bb8f-4846-8c33-281ac51b400c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:13.036759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 113/219] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:13.043761 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b410e-c7f8-4bd0-b475-d3cbb4dba226 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:13.076095 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:13.152965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b410e-c7f8-4bd0-b475-d3cbb4dba226 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:13.190734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e6157-3e30-4818-8733-1490ddccbcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:13.191633 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e6157-3e30-4818-8733-1490ddccbcb1 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:13.192005 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e6157-3e30-4818-8733-1490ddccbcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:13.192293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e6157-3e30-4818-8733-1490ddccbcb1 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:13.193589 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e6157-3e30-4818-8733-1490ddccbcb1 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:13.195959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e6157-3e30-4818-8733-1490ddccbcb1 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:13.198239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 114/220] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:13 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:13.204045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e6157-3e30-4818-8733-1490ddccbcb1 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:13.211687 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 107/221] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:13.249149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608996a0-bb8f-4846-8c33-281ac51b400c None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:13.274582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:13.291533 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:13.332136 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e6157-3e30-4818-8733-1490ddccbcb1 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:13.372154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608996a0-bb8f-4846-8c33-281ac51b400c None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:13.375403 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b2fbe-4e07-4b7b-8c3b-cc81c02fdf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:13.376582 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b2fbe-4e07-4b7b-8c3b-cc81c02fdf85 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:13.377074 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b2fbe-4e07-4b7b-8c3b-cc81c02fdf85 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:13.377570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b2fbe-4e07-4b7b-8c3b-cc81c02fdf85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:13.378576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b2fbe-4e07-4b7b-8c3b-cc81c02fdf85 None admin] RBAC: Adding query filter params (role.id=22f9190503114c4aba5e898e7a460546, scope.project.id=3fa3d431d9cc468aafe7b895f047ff94, user.id=9f1087e8c7e14a5d87c81fdc977088f1) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:13.378980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b2fbe-4e07-4b7b-8c3b-cc81c02fdf85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:13.380657 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b2fbe-4e07-4b7b-8c3b-cc81c02fdf85 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:13.391647 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 115/222] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:13 2022] GET /identity/v3/role_assignments?user.id=9f1087e8c7e14a5d87c81fdc977088f1&scope.project.id=3fa3d431d9cc468aafe7b895f047ff94&role.id=22f9190503114c4aba5e898e7a460546 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:13.394133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f384853-92ca-4b25-ad72-44b18d8f7155 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:13.394712 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f384853-92ca-4b25-ad72-44b18d8f7155 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:13.395261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f384853-92ca-4b25-ad72-44b18d8f7155 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:13.406099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 116/223] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:13.406099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfff9c-aa15-4f8a-b2ce-89fed0adb211 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:13.406099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfff9c-aa15-4f8a-b2ce-89fed0adb211 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:13.406099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfff9c-aa15-4f8a-b2ce-89fed0adb211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:13.427437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-79cfff9c-aa15-4f8a-b2ce-89fed0adb211 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:13.428905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-79cfff9c-aa15-4f8a-b2ce-89fed0adb211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:13.441281 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef6fd2-0fec-4d7a-ae18-3df4cd4400cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:13.442127 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef6fd2-0fec-4d7a-ae18-3df4cd4400cc None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:13.442419 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef6fd2-0fec-4d7a-ae18-3df4cd4400cc None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:13.442699 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef6fd2-0fec-4d7a-ae18-3df4cd4400cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:13.474306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef6fd2-0fec-4d7a-ae18-3df4cd4400cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-6fef6fd2-0fec-4d7a-ae18-3df4cd4400cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:13.483383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:13.484341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:13.485105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:13.485105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:13.485105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:13.485105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:13.485105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:13.485105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:13.485105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:13.485105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:13.485105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:13.485105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:13.485105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:38:13.485105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:13.493307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 108/224] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:38:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:38:13.499679 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef6fd2-0fec-4d7a-ae18-3df4cd4400cc None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:13.507762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 117/225] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:13.532042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfff9c-aa15-4f8a-b2ce-89fed0adb211 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:13.615505 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:13.651181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfff9c-aa15-4f8a-b2ce-89fed0adb211 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:13.675533 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:13.695706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28b624-4d10-497a-98f4-a074eabc10e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:13.697168 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28b624-4d10-497a-98f4-a074eabc10e4 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:13.697667 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28b624-4d10-497a-98f4-a074eabc10e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:13.698183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28b624-4d10-497a-98f4-a074eabc10e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:13.708949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef6fd2-0fec-4d7a-ae18-3df4cd4400cc None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:13.711065 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28b624-4d10-497a-98f4-a074eabc10e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-bc28b624-4d10-497a-98f4-a074eabc10e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:13.719944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:13.720798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:13.721546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:13.721546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:13.721546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:13.721546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:13.721546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:13.721546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:13.721546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:13.721546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:13.721546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:13.721546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:13.721546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:13.721546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:13.724479 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 118/226] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:38:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:13.731037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28b624-4d10-497a-98f4-a074eabc10e4 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:13.769584 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd32be1-462c-4d80-9c36-b02eb099e236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:13.770789 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd32be1-462c-4d80-9c36-b02eb099e236 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:13.770789 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd32be1-462c-4d80-9c36-b02eb099e236 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:13.770955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd32be1-462c-4d80-9c36-b02eb099e236 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:13.771697 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd32be1-462c-4d80-9c36-b02eb099e236 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:13.772030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd32be1-462c-4d80-9c36-b02eb099e236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:13.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd32be1-462c-4d80-9c36-b02eb099e236 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:13.791590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 109/227] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:13.797367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd32be1-462c-4d80-9c36-b02eb099e236 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:13.844560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28b624-4d10-497a-98f4-a074eabc10e4 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:13.880119 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:13.911705 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff599d-aa93-4d47-934c-8bcbea7bda74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:13.912493 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff599d-aa93-4d47-934c-8bcbea7bda74 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:13.912778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff599d-aa93-4d47-934c-8bcbea7bda74 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:13.913058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff599d-aa93-4d47-934c-8bcbea7bda74 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:13.914030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff599d-aa93-4d47-934c-8bcbea7bda74 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:13.914485 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff599d-aa93-4d47-934c-8bcbea7bda74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:13.915953 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff599d-aa93-4d47-934c-8bcbea7bda74 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:13.942720 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 119/228] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:13.949175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff599d-aa93-4d47-934c-8bcbea7bda74 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:14.003037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd32be1-462c-4d80-9c36-b02eb099e236 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:14.056238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a020d0-a6c1-4435-bcde-8adf2e4a14e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:14.057154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a020d0-a6c1-4435-bcde-8adf2e4a14e1 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:14.057905 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a020d0-a6c1-4435-bcde-8adf2e4a14e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:14.058223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a020d0-a6c1-4435-bcde-8adf2e4a14e1 None admin] PATH_INFO: `/v3/users/7805fa8ea7214ce2b5c934735e50d7ca` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:14.059824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a020d0-a6c1-4435-bcde-8adf2e4a14e1 None admin] RBAC: Authorizing `identity:get_user(user_id=7805fa8ea7214ce2b5c934735e50d7ca)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:14.061824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a020d0-a6c1-4435-bcde-8adf2e4a14e1 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:14.064307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 110/229] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:13 2022] GET /identity/v3/users/7805fa8ea7214ce2b5c934735e50d7ca => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:14.069962 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a020d0-a6c1-4435-bcde-8adf2e4a14e1 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:14.073934 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:14.123068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff599d-aa93-4d47-934c-8bcbea7bda74 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:14.180480 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def96d94-8171-465f-a55c-a872d4ab34d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:14.181757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def96d94-8171-465f-a55c-a872d4ab34d5 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:14.182247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def96d94-8171-465f-a55c-a872d4ab34d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:14.182724 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def96d94-8171-465f-a55c-a872d4ab34d5 None admin] PATH_INFO: `/v3/users/7805fa8ea7214ce2b5c934735e50d7ca` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:14.184278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def96d94-8171-465f-a55c-a872d4ab34d5 None admin] RBAC: Authorizing `identity:get_user(user_id=7805fa8ea7214ce2b5c934735e50d7ca)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:14.185951 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def96d94-8171-465f-a55c-a872d4ab34d5 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:14.187874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 120/230] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:13 2022] GET /identity/v3/users/7805fa8ea7214ce2b5c934735e50d7ca => generated 283 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:14.193363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def96d94-8171-465f-a55c-a872d4ab34d5 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:14.222988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a020d0-a6c1-4435-bcde-8adf2e4a14e1 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:14.263508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeb1e1-229f-441a-b7ff-088d21449490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:14.265043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeb1e1-229f-441a-b7ff-088d21449490 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:14.265586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeb1e1-229f-441a-b7ff-088d21449490 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:14.266893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeb1e1-229f-441a-b7ff-088d21449490 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:14.268093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebeb1e1-229f-441a-b7ff-088d21449490 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:14.269639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebeb1e1-229f-441a-b7ff-088d21449490 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:14.271449 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 111/231] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:14 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:14.283744 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeb1e1-229f-441a-b7ff-088d21449490 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:14.311052 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:14.360105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:14.398586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeb1e1-229f-441a-b7ff-088d21449490 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:14.419987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def96d94-8171-465f-a55c-a872d4ab34d5 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:14.437208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339e3bb-499c-4115-87f5-5c0d50f3229e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:14.438124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339e3bb-499c-4115-87f5-5c0d50f3229e None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:14.438407 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339e3bb-499c-4115-87f5-5c0d50f3229e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:14.438681 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339e3bb-499c-4115-87f5-5c0d50f3229e None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94/users/7805fa8ea7214ce2b5c934735e50d7ca/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:14.441207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339e3bb-499c-4115-87f5-5c0d50f3229e None admin] RBAC: Authorizing `identity:create_grant(project_id=3fa3d431d9cc468aafe7b895f047ff94, user_id=7805fa8ea7214ce2b5c934735e50d7ca, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:14.442937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339e3bb-499c-4115-87f5-5c0d50f3229e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:14.459827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 112/232] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:38:14 2022] PUT /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94/users/7805fa8ea7214ce2b5c934735e50d7ca/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:38:14.511997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe1668-60bd-4444-b7df-5f14cc70d117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:14.512871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe1668-60bd-4444-b7df-5f14cc70d117 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:14.513158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe1668-60bd-4444-b7df-5f14cc70d117 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:14.513437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe1668-60bd-4444-b7df-5f14cc70d117 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:14.514703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe1668-60bd-4444-b7df-5f14cc70d117 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:14.516308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe1668-60bd-4444-b7df-5f14cc70d117 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:14.518563 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 121/233] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:14 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:14.525909 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339e3bb-499c-4115-87f5-5c0d50f3229e None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:14.659158 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339e3bb-499c-4115-87f5-5c0d50f3229e None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:14.692107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:14.697791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5bd73-8dd4-4997-bf87-cff05bbfdb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:14.698710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5bd73-8dd4-4997-bf87-cff05bbfdb3a None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:14.699206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5bd73-8dd4-4997-bf87-cff05bbfdb3a None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:14.699801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5bd73-8dd4-4997-bf87-cff05bbfdb3a None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94/users/7805fa8ea7214ce2b5c934735e50d7ca/roles/04597f92ac654fb4ac68dbb783d3dc31` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:14.703102 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5bd73-8dd4-4997-bf87-cff05bbfdb3a None admin] RBAC: Authorizing `identity:create_grant(project_id=3fa3d431d9cc468aafe7b895f047ff94, user_id=7805fa8ea7214ce2b5c934735e50d7ca, role_id=04597f92ac654fb4ac68dbb783d3dc31)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:14.704592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5bd73-8dd4-4997-bf87-cff05bbfdb3a None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:14.724628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 113/234] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:38:14 2022] PUT /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94/users/7805fa8ea7214ce2b5c934735e50d7ca/roles/04597f92ac654fb4ac68dbb783d3dc31 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:38:15.117676 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d493bd-c420-419d-b3e5-577dedf5848d None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:15.118152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d493bd-c420-419d-b3e5-577dedf5848d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:15.118549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d493bd-c420-419d-b3e5-577dedf5848d None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:15.120425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 122/235] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:15.126008 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b0f94-8875-4310-b315-0716f19ad874 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:15.126481 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b0f94-8875-4310-b315-0716f19ad874 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:15.126552 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b0f94-8875-4310-b315-0716f19ad874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:15.145014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-609b0f94-8875-4310-b315-0716f19ad874 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:15.146223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-609b0f94-8875-4310-b315-0716f19ad874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:15.223912 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3abcf1-5f02-45a1-b73d-ea132c13ac28 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:15.224411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3abcf1-5f02-45a1-b73d-ea132c13ac28 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:15.224480 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3abcf1-5f02-45a1-b73d-ea132c13ac28 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:15.229938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 123/236] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:15.237214 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c860cb-72ce-4d34-bbed-bede0050deea None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:15.237547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c860cb-72ce-4d34-bbed-bede0050deea None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:15.237735 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 114/237] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:15.237847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c860cb-72ce-4d34-bbed-bede0050deea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:15.268788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-32c860cb-72ce-4d34-bbed-bede0050deea None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:15.270088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-32c860cb-72ce-4d34-bbed-bede0050deea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:15.297884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b0f94-8875-4310-b315-0716f19ad874 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:15.406623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 124/238] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:15.443093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c860cb-72ce-4d34-bbed-bede0050deea None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:15.488750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b0f94-8875-4310-b315-0716f19ad874 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:15.541385 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:15.555440 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae7979-5f68-4f0a-a5fb-eae82075426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:15.557788 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae7979-5f68-4f0a-a5fb-eae82075426c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:15.562137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae7979-5f68-4f0a-a5fb-eae82075426c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:15.562572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae7979-5f68-4f0a-a5fb-eae82075426c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:15.564088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae7979-5f68-4f0a-a5fb-eae82075426c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:15.565717 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae7979-5f68-4f0a-a5fb-eae82075426c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:15.570242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 115/239] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:15.572435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:15.577398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae7979-5f68-4f0a-a5fb-eae82075426c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:15.670217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c860cb-72ce-4d34-bbed-bede0050deea None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:15.734600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bb233-792e-44dc-a210-fb575284eaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:15.735411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bb233-792e-44dc-a210-fb575284eaf3 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:15.735697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bb233-792e-44dc-a210-fb575284eaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:15.735994 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bb233-792e-44dc-a210-fb575284eaf3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:15.764881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bb233-792e-44dc-a210-fb575284eaf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-cd9bb233-792e-44dc-a210-fb575284eaf3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:15.784070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:15.785212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:15.787007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:15.787007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:15.787007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:15.787007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:15.787007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:15.787007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:15.787007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:15.787007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:15.787007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:15.787007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:15.787007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:15.787007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:15.787007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 125/240] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:38:15.793503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae7979-5f68-4f0a-a5fb-eae82075426c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:15.795736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bb233-792e-44dc-a210-fb575284eaf3 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:15.874306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f25ab9-48ba-4693-ad3b-3328616df1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:15.875052 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f25ab9-48ba-4693-ad3b-3328616df1d0 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:15.875339 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f25ab9-48ba-4693-ad3b-3328616df1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:15.877259 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f25ab9-48ba-4693-ad3b-3328616df1d0 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:15.878027 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f25ab9-48ba-4693-ad3b-3328616df1d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:15.879534 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f25ab9-48ba-4693-ad3b-3328616df1d0 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:15.920028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 116/241] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:38:15 2022] POST /identity/v3/users => generated 293 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:38:15.971881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bb233-792e-44dc-a210-fb575284eaf3 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:15.987833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:16.013209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5832e65-ea13-4592-af2c-1d3e79a4e436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:16.013959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5832e65-ea13-4592-af2c-1d3e79a4e436 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:16.014251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5832e65-ea13-4592-af2c-1d3e79a4e436 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:16.014586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5832e65-ea13-4592-af2c-1d3e79a4e436 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:16.015366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5832e65-ea13-4592-af2c-1d3e79a4e436 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:16.015741 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5832e65-ea13-4592-af2c-1d3e79a4e436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:16.017235 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5832e65-ea13-4592-af2c-1d3e79a4e436 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:16.028225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 126/242] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:16.034383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f25ab9-48ba-4693-ad3b-3328616df1d0 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:16.106227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:16.188093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f25ab9-48ba-4693-ad3b-3328616df1d0 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:16.214922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03535cef-5f16-4ab2-a584-1722b039b42e None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:16.215317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03535cef-5f16-4ab2-a584-1722b039b42e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:16.215546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03535cef-5f16-4ab2-a584-1722b039b42e None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:16.217727 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 127/243] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:16.223896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1426f9b-d318-4069-a3ec-ca8be17e573e None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:16.224158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1426f9b-d318-4069-a3ec-ca8be17e573e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:16.224681 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1426f9b-d318-4069-a3ec-ca8be17e573e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:16.245474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f81872-b5d9-4001-b502-247c34fa6b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:16.246620 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f81872-b5d9-4001-b502-247c34fa6b7a None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:16.246899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f81872-b5d9-4001-b502-247c34fa6b7a None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:16.247171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f81872-b5d9-4001-b502-247c34fa6b7a None admin] PATH_INFO: `/v3/users/dad214f06354464091b9b8268e2b4ca0` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:16.260510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-c1426f9b-d318-4069-a3ec-ca8be17e573e None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:16.262034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c1426f9b-d318-4069-a3ec-ca8be17e573e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:16.269433 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f81872-b5d9-4001-b502-247c34fa6b7a None admin] RBAC: Authorizing `identity:get_user(user_id=dad214f06354464091b9b8268e2b4ca0)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:16.271221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f81872-b5d9-4001-b502-247c34fa6b7a None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:16.275018 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 117/244] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:16 2022] GET /identity/v3/users/dad214f06354464091b9b8268e2b4ca0 => generated 297 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:16.280661 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f81872-b5d9-4001-b502-247c34fa6b7a None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:16.373762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 128/245] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:16.400574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1426f9b-d318-4069-a3ec-ca8be17e573e None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:16.433502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f81872-b5d9-4001-b502-247c34fa6b7a None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:16.437132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:16.487179 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cd849-b088-4e5c-9fae-73e2c2ec8504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:16.488510 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cd849-b088-4e5c-9fae-73e2c2ec8504 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:16.490853 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cd849-b088-4e5c-9fae-73e2c2ec8504 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:16.490853 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cd849-b088-4e5c-9fae-73e2c2ec8504 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:16.555077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1426f9b-d318-4069-a3ec-ca8be17e573e None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:16.565668 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:16.595790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23cd849-b088-4e5c-9fae-73e2c2ec8504 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:16.598611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23cd849-b088-4e5c-9fae-73e2c2ec8504 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:16.602488 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d441716-2356-4f2a-92ef-5ca1560cb73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:16.603300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d441716-2356-4f2a-92ef-5ca1560cb73b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:16.603585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d441716-2356-4f2a-92ef-5ca1560cb73b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:16.603922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d441716-2356-4f2a-92ef-5ca1560cb73b None admin] PATH_INFO: `/v3/roles/member` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:16.618411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d441716-2356-4f2a-92ef-5ca1560cb73b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-3d441716-2356-4f2a-92ef-5ca1560cb73b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:16.628344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:16.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:16.630103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:16.630103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:16.630103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:16.630103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:16.630103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:16.630103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:16.630103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:16.630103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:16.630103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:16.630103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:16.630103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:16.630103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:16.633884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 129/246] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:16.637446 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d441716-2356-4f2a-92ef-5ca1560cb73b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:16.767236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 118/247] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 20:38:16.776801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cd849-b088-4e5c-9fae-73e2c2ec8504 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:16.781178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:16.814837 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d441716-2356-4f2a-92ef-5ca1560cb73b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:16.884985 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6119db6b-aae7-4f79-9d9e-2d3fc495fff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:16.885707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6119db6b-aae7-4f79-9d9e-2d3fc495fff5 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:16.885991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6119db6b-aae7-4f79-9d9e-2d3fc495fff5 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:16.886270 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6119db6b-aae7-4f79-9d9e-2d3fc495fff5 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:16.887039 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6119db6b-aae7-4f79-9d9e-2d3fc495fff5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:16.887352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6119db6b-aae7-4f79-9d9e-2d3fc495fff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:16.888801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6119db6b-aae7-4f79-9d9e-2d3fc495fff5 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:16.903106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 130/248] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:16.911028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6119db6b-aae7-4f79-9d9e-2d3fc495fff5 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:16.965547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:16.967043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cd849-b088-4e5c-9fae-73e2c2ec8504 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:17.044435 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0357bd-031a-43c0-85be-08c4e907b642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:17.045241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0357bd-031a-43c0-85be-08c4e907b642 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:17.045530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0357bd-031a-43c0-85be-08c4e907b642 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:17.045812 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0357bd-031a-43c0-85be-08c4e907b642 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:17.048981 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0357bd-031a-43c0-85be-08c4e907b642 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:17.050754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0357bd-031a-43c0-85be-08c4e907b642 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:17.054300 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 119/249] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:16 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:17.060811 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2d35d-f967-468d-a324-ad2b3517b78f None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:17.062236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2d35d-f967-468d-a324-ad2b3517b78f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:17.065460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2d35d-f967-468d-a324-ad2b3517b78f None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:17.067447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 120/250] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:17.069129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2d35d-f967-468d-a324-ad2b3517b78f None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:17.126030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6119db6b-aae7-4f79-9d9e-2d3fc495fff5 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:17.179627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:17.202612 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb01014-9f18-4482-acb5-977870d8f9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:17.203946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb01014-9f18-4482-acb5-977870d8f9d5 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:17.204428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb01014-9f18-4482-acb5-977870d8f9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:17.204911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb01014-9f18-4482-acb5-977870d8f9d5 None admin] PATH_INFO: `/v3/users/9f1087e8c7e14a5d87c81fdc977088f1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:17.206391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb01014-9f18-4482-acb5-977870d8f9d5 None admin] RBAC: Authorizing `identity:get_user(user_id=9f1087e8c7e14a5d87c81fdc977088f1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:17.208223 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb01014-9f18-4482-acb5-977870d8f9d5 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:17.210233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 131/251] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:16 2022] GET /identity/v3/users/9f1087e8c7e14a5d87c81fdc977088f1 => generated 297 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:17.211778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1d09d-1817-4d2a-933b-b8a947032598 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:17.212267 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1d09d-1817-4d2a-933b-b8a947032598 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:17.212750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1d09d-1817-4d2a-933b-b8a947032598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:17.225577 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2d35d-f967-468d-a324-ad2b3517b78f None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:17.232473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-05d1d09d-1817-4d2a-933b-b8a947032598 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:17.234413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-05d1d09d-1817-4d2a-933b-b8a947032598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:17.263460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45020544-b2c7-4276-b80c-15aaf2c76c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:17.264243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45020544-b2c7-4276-b80c-15aaf2c76c99 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:17.264466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45020544-b2c7-4276-b80c-15aaf2c76c99 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:17.264740 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45020544-b2c7-4276-b80c-15aaf2c76c99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:17.265961 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45020544-b2c7-4276-b80c-15aaf2c76c99 None admin] RBAC: Adding query filter params (role.id=1052f958732d4259a0658b40d1f9c193, scope.project.id=3fa3d431d9cc468aafe7b895f047ff94, user.id=dad214f06354464091b9b8268e2b4ca0) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:17.266068 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45020544-b2c7-4276-b80c-15aaf2c76c99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:17.267539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45020544-b2c7-4276-b80c-15aaf2c76c99 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:17.273856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 121/252] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:17 2022] GET /identity/v3/role_assignments?user.id=dad214f06354464091b9b8268e2b4ca0&scope.project.id=3fa3d431d9cc468aafe7b895f047ff94&role.id=1052f958732d4259a0658b40d1f9c193 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:17.274785 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45020544-b2c7-4276-b80c-15aaf2c76c99 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:17.354869 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 132/253] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:17.392274 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1d09d-1817-4d2a-933b-b8a947032598 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:17.441591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:17.521926 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45020544-b2c7-4276-b80c-15aaf2c76c99 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:17.587960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e390c5c-f933-425c-baa5-733efe8774c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:17.589202 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e390c5c-f933-425c-baa5-733efe8774c5 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:17.589681 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e390c5c-f933-425c-baa5-733efe8774c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:17.590159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e390c5c-f933-425c-baa5-733efe8774c5 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:17.591647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e390c5c-f933-425c-baa5-733efe8774c5 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:17.593955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e390c5c-f933-425c-baa5-733efe8774c5 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:17.597291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 122/254] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:17 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:17.609057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e390c5c-f933-425c-baa5-733efe8774c5 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:17.617878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1d09d-1817-4d2a-933b-b8a947032598 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:17.628090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:17.704814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f443432-2c92-4a08-a5f5-6256e5af9bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:17.706113 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f443432-2c92-4a08-a5f5-6256e5af9bbb None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:17.706613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f443432-2c92-4a08-a5f5-6256e5af9bbb None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:17.707104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f443432-2c92-4a08-a5f5-6256e5af9bbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:17.723987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f443432-2c92-4a08-a5f5-6256e5af9bbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-3f443432-2c92-4a08-a5f5-6256e5af9bbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:17.733607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:17.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:17.735293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:17.735293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:17.735293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:17.735293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:17.735293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:17.735293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:17.735293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:17.735293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:17.735293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:17.735293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:17.735293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:38:17.735293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:17.735293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 133/255] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:38:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:38:17.741102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f443432-2c92-4a08-a5f5-6256e5af9bbb None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:17.815484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e390c5c-f933-425c-baa5-733efe8774c5 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:17.868859 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89dd3e-7977-4cdf-b28f-1aa4639c79df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:17.869690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89dd3e-7977-4cdf-b28f-1aa4639c79df None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:17.869972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89dd3e-7977-4cdf-b28f-1aa4639c79df None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:17.870246 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89dd3e-7977-4cdf-b28f-1aa4639c79df None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94/users/9f1087e8c7e14a5d87c81fdc977088f1/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:17.875054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89dd3e-7977-4cdf-b28f-1aa4639c79df None admin] RBAC: Authorizing `identity:create_grant(project_id=3fa3d431d9cc468aafe7b895f047ff94, user_id=9f1087e8c7e14a5d87c81fdc977088f1, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:17.876561 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89dd3e-7977-4cdf-b28f-1aa4639c79df None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:17.908546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 123/256] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:38:17 2022] PUT /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94/users/9f1087e8c7e14a5d87c81fdc977088f1/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:38:17.914472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f71fb-6d6e-4986-985b-b5f40c1f967a None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:17.914780 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f71fb-6d6e-4986-985b-b5f40c1f967a None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:17.916484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f71fb-6d6e-4986-985b-b5f40c1f967a None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:17.924278 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 124/257] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:17 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:17.969354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:17.975276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84834496-0a83-404f-bb78-4af52e834247 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:17.975583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84834496-0a83-404f-bb78-4af52e834247 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:17.979742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84834496-0a83-404f-bb78-4af52e834247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:18.015772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f443432-2c92-4a08-a5f5-6256e5af9bbb None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:18.017756 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-84834496-0a83-404f-bb78-4af52e834247 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:18.019110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-84834496-0a83-404f-bb78-4af52e834247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:18.069802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d02852-c968-4732-8d6e-2a08102922f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:18.070784 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d02852-c968-4732-8d6e-2a08102922f2 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:18.071069 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d02852-c968-4732-8d6e-2a08102922f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:18.073690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d02852-c968-4732-8d6e-2a08102922f2 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:18.074517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d02852-c968-4732-8d6e-2a08102922f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:18.074829 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d02852-c968-4732-8d6e-2a08102922f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:18.076294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d02852-c968-4732-8d6e-2a08102922f2 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:18.105234 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 134/258] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:18.113112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d02852-c968-4732-8d6e-2a08102922f2 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:18.155185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:18.175106 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 125/259] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:18.201255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84834496-0a83-404f-bb78-4af52e834247 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:18.315823 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d02852-c968-4732-8d6e-2a08102922f2 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:18.369016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6440bd4-1124-4945-90f8-6b2d0722191e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:18.369910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6440bd4-1124-4945-90f8-6b2d0722191e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:18.370193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6440bd4-1124-4945-90f8-6b2d0722191e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:18.370468 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6440bd4-1124-4945-90f8-6b2d0722191e None admin] PATH_INFO: `/v3/users/7805fa8ea7214ce2b5c934735e50d7ca` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:18.376493 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6440bd4-1124-4945-90f8-6b2d0722191e None admin] RBAC: Authorizing `identity:get_user(user_id=7805fa8ea7214ce2b5c934735e50d7ca)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:18.377981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6440bd4-1124-4945-90f8-6b2d0722191e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:18.379742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 135/260] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:18 2022] GET /identity/v3/users/7805fa8ea7214ce2b5c934735e50d7ca => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:18.385529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6440bd4-1124-4945-90f8-6b2d0722191e None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:18.413950 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84834496-0a83-404f-bb78-4af52e834247 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:18.448135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:18.540739 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24daad38-a522-4d09-9118-a6e80d14f035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:18.541599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daad38-a522-4d09-9118-a6e80d14f035 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:18.541887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daad38-a522-4d09-9118-a6e80d14f035 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:18.542164 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daad38-a522-4d09-9118-a6e80d14f035 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:18.565837 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6440bd4-1124-4945-90f8-6b2d0722191e None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:18.580388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24daad38-a522-4d09-9118-a6e80d14f035 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-24daad38-a522-4d09-9118-a6e80d14f035 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:18.588770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:18.589753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:18.590501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:18.590501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:18.590501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:18.590501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:18.590501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:18.590501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:18.590501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:18.590501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:18.590501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:18.590501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:18.590501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:18.590501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:18.596076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 126/261] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:38:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:38:18.596995 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24daad38-a522-4d09-9118-a6e80d14f035 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:18.617115 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73900e-6b11-4134-8b47-89a7948a7b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:18.618187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73900e-6b11-4134-8b47-89a7948a7b2b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:18.618483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73900e-6b11-4134-8b47-89a7948a7b2b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:18.618762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73900e-6b11-4134-8b47-89a7948a7b2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:18.744755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:18.780577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73900e-6b11-4134-8b47-89a7948a7b2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:18.782072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73900e-6b11-4134-8b47-89a7948a7b2b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:18.817024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:18.839355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24daad38-a522-4d09-9118-a6e80d14f035 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:18.913683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93747b3-afcd-48df-bfd6-2fdb6ae4bcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:18.914828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93747b3-afcd-48df-bfd6-2fdb6ae4bcbc None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:18.915399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93747b3-afcd-48df-bfd6-2fdb6ae4bcbc None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:18.916627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93747b3-afcd-48df-bfd6-2fdb6ae4bcbc None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:18.917607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93747b3-afcd-48df-bfd6-2fdb6ae4bcbc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:18.918472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93747b3-afcd-48df-bfd6-2fdb6ae4bcbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:18.920151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93747b3-afcd-48df-bfd6-2fdb6ae4bcbc None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:18.941058 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 127/262] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:18.947077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93747b3-afcd-48df-bfd6-2fdb6ae4bcbc None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:18.985889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 136/263] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 20:38:18.994229 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73900e-6b11-4134-8b47-89a7948a7b2b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:19.174704 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93747b3-afcd-48df-bfd6-2fdb6ae4bcbc None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:19.215254 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73900e-6b11-4134-8b47-89a7948a7b2b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:19.219840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:19.253777 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:19.257495 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4f75d-92bc-420e-ad87-10760636855f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:19.261734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4f75d-92bc-420e-ad87-10760636855f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:19.262330 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4f75d-92bc-420e-ad87-10760636855f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:19.262823 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4f75d-92bc-420e-ad87-10760636855f None admin] PATH_INFO: `/v3/users/7805fa8ea7214ce2b5c934735e50d7ca` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:19.265386 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4f75d-92bc-420e-ad87-10760636855f None admin] RBAC: Authorizing `identity:get_user(user_id=7805fa8ea7214ce2b5c934735e50d7ca)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:19.267061 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4f75d-92bc-420e-ad87-10760636855f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:19.271150 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 128/264] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:18 2022] GET /identity/v3/users/7805fa8ea7214ce2b5c934735e50d7ca => generated 283 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:19.276283 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4f75d-92bc-420e-ad87-10760636855f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:19.303090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ecfb7-7774-47f3-95b0-5ab27db9f386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:19.308654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ecfb7-7774-47f3-95b0-5ab27db9f386 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:19.309241 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ecfb7-7774-47f3-95b0-5ab27db9f386 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:19.309739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ecfb7-7774-47f3-95b0-5ab27db9f386 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:19.313070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ecfb7-7774-47f3-95b0-5ab27db9f386 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:19.318827 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ecfb7-7774-47f3-95b0-5ab27db9f386 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:19.322569 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 137/265] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:18 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:19.324176 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69359bc-0df6-4d36-beb6-5a7b15bcf270 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:19.324700 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69359bc-0df6-4d36-beb6-5a7b15bcf270 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:19.325167 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69359bc-0df6-4d36-beb6-5a7b15bcf270 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:19.326814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 138/266] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:19.345239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed45ad0-5d83-46df-b850-93167dd751cb None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:19.345904 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed45ad0-5d83-46df-b850-93167dd751cb None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:19.346500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed45ad0-5d83-46df-b850-93167dd751cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:19.378145 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-6ed45ad0-5d83-46df-b850-93167dd751cb None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:19.380307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6ed45ad0-5d83-46df-b850-93167dd751cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:19.445277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4f75d-92bc-420e-ad87-10760636855f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:19.474034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 139/267] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:38:19.475438 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed45ad0-5d83-46df-b850-93167dd751cb None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:19.491993 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e7476-c501-4fb4-ade4-039610f17dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:19.492655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e7476-c501-4fb4-ade4-039610f17dc8 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:19.492935 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e7476-c501-4fb4-ade4-039610f17dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:19.493266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e7476-c501-4fb4-ade4-039610f17dc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:19.557696 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:19.581932 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e7476-c501-4fb4-ade4-039610f17dc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:19.584379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e7476-c501-4fb4-ade4-039610f17dc8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:19.624473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed45ad0-5d83-46df-b850-93167dd751cb None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:19.644763 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:19.676244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fe6d4-983b-4455-9916-561f2692a337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:19.677584 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fe6d4-983b-4455-9916-561f2692a337 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:19.678073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fe6d4-983b-4455-9916-561f2692a337 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:19.678539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fe6d4-983b-4455-9916-561f2692a337 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:19.679986 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fe6d4-983b-4455-9916-561f2692a337 None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=3fa3d431d9cc468aafe7b895f047ff94, user.id=7805fa8ea7214ce2b5c934735e50d7ca) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:19.680689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fe6d4-983b-4455-9916-561f2692a337 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:19.683300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fe6d4-983b-4455-9916-561f2692a337 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:19.691974 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 140/268] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:19 2022] GET /identity/v3/role_assignments?user.id=7805fa8ea7214ce2b5c934735e50d7ca&scope.project.id=3fa3d431d9cc468aafe7b895f047ff94&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:19.693563 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69744e00-4d4f-4eb8-ae6d-d5c024bdfa97 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:19.694051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69744e00-4d4f-4eb8-ae6d-d5c024bdfa97 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:19.694594 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69744e00-4d4f-4eb8-ae6d-d5c024bdfa97 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:19.697806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 141/269] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:19.698987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69744e00-4d4f-4eb8-ae6d-d5c024bdfa97 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:19.765456 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 129/270] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:19.768133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b373a-7ebf-4ba7-9240-5167a9263881 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:19.770740 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b373a-7ebf-4ba7-9240-5167a9263881 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:19.772189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b373a-7ebf-4ba7-9240-5167a9263881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:19.806930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-8c4b373a-7ebf-4ba7-9240-5167a9263881 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:19.808713 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8c4b373a-7ebf-4ba7-9240-5167a9263881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:19.893610 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 130/271] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:19.894875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b373a-7ebf-4ba7-9240-5167a9263881 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:19.933311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:19.966841 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69744e00-4d4f-4eb8-ae6d-d5c024bdfa97 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:20.009317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da934f6a-8cb0-48ff-a099-23a77248bde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:20.010521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da934f6a-8cb0-48ff-a099-23a77248bde5 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:20.011009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da934f6a-8cb0-48ff-a099-23a77248bde5 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:20.011507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da934f6a-8cb0-48ff-a099-23a77248bde5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:20.013064 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da934f6a-8cb0-48ff-a099-23a77248bde5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:20.014807 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da934f6a-8cb0-48ff-a099-23a77248bde5 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:20.016871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 142/272] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:20.017970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da934f6a-8cb0-48ff-a099-23a77248bde5 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:20.103260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:20.111498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b373a-7ebf-4ba7-9240-5167a9263881 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:20.179056 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be08f4-7b16-4885-9013-11ef3178c792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:20.180815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be08f4-7b16-4885-9013-11ef3178c792 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:20.181317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be08f4-7b16-4885-9013-11ef3178c792 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:20.181815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be08f4-7b16-4885-9013-11ef3178c792 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:20.183305 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be08f4-7b16-4885-9013-11ef3178c792 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:20.185690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be08f4-7b16-4885-9013-11ef3178c792 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:20.187951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 131/273] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:19 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:20.189484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be08f4-7b16-4885-9013-11ef3178c792 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:20.195451 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da934f6a-8cb0-48ff-a099-23a77248bde5 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:20.240216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf17a35-6083-4953-8835-2272fc8b0d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:20.242102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf17a35-6083-4953-8835-2272fc8b0d03 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:20.242874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf17a35-6083-4953-8835-2272fc8b0d03 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:20.243634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf17a35-6083-4953-8835-2272fc8b0d03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:20.262153 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf17a35-6083-4953-8835-2272fc8b0d03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-2bf17a35-6083-4953-8835-2272fc8b0d03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:20.277920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:20.279809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:20.281321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:20.281321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:20.281321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:20.281321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:20.281321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:20.281321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:20.281321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:20.281321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:20.281321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:20.281321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:20.281321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:20.281321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:20.285596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 143/274] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:20.292749 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf17a35-6083-4953-8835-2272fc8b0d03 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:20.296049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:20.316522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be08f4-7b16-4885-9013-11ef3178c792 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:20.376649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a764be2-ce62-40ca-8539-7786d8a72376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:20.377344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a764be2-ce62-40ca-8539-7786d8a72376 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:20.377630 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a764be2-ce62-40ca-8539-7786d8a72376 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:20.377955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a764be2-ce62-40ca-8539-7786d8a72376 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:20.378810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a764be2-ce62-40ca-8539-7786d8a72376 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:20.380327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a764be2-ce62-40ca-8539-7786d8a72376 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:20.413411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 132/275] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:38:20 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:38:20.414763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a764be2-ce62-40ca-8539-7786d8a72376 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:20.467770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf17a35-6083-4953-8835-2272fc8b0d03 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:20.483396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:20.517391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53db85-7ab2-4ee9-ba40-912ae84888ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:20.518073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53db85-7ab2-4ee9-ba40-912ae84888ef None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:20.518428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53db85-7ab2-4ee9-ba40-912ae84888ef None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:20.518776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53db85-7ab2-4ee9-ba40-912ae84888ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:20.519739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53db85-7ab2-4ee9-ba40-912ae84888ef None admin] RBAC: Adding query filter params (role.id=04597f92ac654fb4ac68dbb783d3dc31, scope.project.id=3fa3d431d9cc468aafe7b895f047ff94, user.id=7805fa8ea7214ce2b5c934735e50d7ca) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:20.520142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53db85-7ab2-4ee9-ba40-912ae84888ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:20.521835 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53db85-7ab2-4ee9-ba40-912ae84888ef None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:20.529797 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 144/276] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:20 2022] GET /identity/v3/role_assignments?user.id=7805fa8ea7214ce2b5c934735e50d7ca&scope.project.id=3fa3d431d9cc468aafe7b895f047ff94&role.id=04597f92ac654fb4ac68dbb783d3dc31 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:20.537639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ff5b5-cd47-456e-b5ce-4bf00b4be7df None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:20.538024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ff5b5-cd47-456e-b5ce-4bf00b4be7df None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:20.538326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ff5b5-cd47-456e-b5ce-4bf00b4be7df None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:20.544124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 145/277] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:20 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:20.558062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19625f12-076a-4292-9bcf-f25e6208172a None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:20.558359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19625f12-076a-4292-9bcf-f25e6208172a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:20.558782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19625f12-076a-4292-9bcf-f25e6208172a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:20.567193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a764be2-ce62-40ca-8539-7786d8a72376 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:20.596915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-19625f12-076a-4292-9bcf-f25e6208172a None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:20.598348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-19625f12-076a-4292-9bcf-f25e6208172a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:20.613090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c0c7e-b821-4216-98e5-1065f6cd3e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:20.614853 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c0c7e-b821-4216-98e5-1065f6cd3e69 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:20.615595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c0c7e-b821-4216-98e5-1065f6cd3e69 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:20.616391 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c0c7e-b821-4216-98e5-1065f6cd3e69 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:20.617779 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c0c7e-b821-4216-98e5-1065f6cd3e69 None admin] RBAC: Adding query filter params (name=member) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:20.618313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c0c7e-b821-4216-98e5-1065f6cd3e69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:20.620119 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c0c7e-b821-4216-98e5-1065f6cd3e69 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:20.626467 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:20.633745 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 133/278] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:20.642549 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c0c7e-b821-4216-98e5-1065f6cd3e69 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:20.705181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 146/279] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:38:20.708570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a1cdd-0427-4dac-92b6-8ada2439ecba None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:20.709347 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a1cdd-0427-4dac-92b6-8ada2439ecba None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:20.709621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a1cdd-0427-4dac-92b6-8ada2439ecba None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:20.711473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 147/280] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:20.717473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee6d78-a978-4691-935a-0682d731c44d None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:20.717803 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee6d78-a978-4691-935a-0682d731c44d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:20.718082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee6d78-a978-4691-935a-0682d731c44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:20.745683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-82ee6d78-a978-4691-935a-0682d731c44d None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:20.746944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-82ee6d78-a978-4691-935a-0682d731c44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:20.835314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c0c7e-b821-4216-98e5-1065f6cd3e69 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:20.841475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 148/281] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:20.842489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee6d78-a978-4691-935a-0682d731c44d None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:20.878501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:20.881429 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0ca23-1a4d-4e7b-a34c-98f26c3a5540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:20.882794 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0ca23-1a4d-4e7b-a34c-98f26c3a5540 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:20.883240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0ca23-1a4d-4e7b-a34c-98f26c3a5540 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:20.883534 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0ca23-1a4d-4e7b-a34c-98f26c3a5540 None admin] PATH_INFO: `/v3/users/1993525339404ca5b5aed2d3902cf643` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:20.901448 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0ca23-1a4d-4e7b-a34c-98f26c3a5540 None admin] RBAC: Authorizing `identity:get_user(user_id=1993525339404ca5b5aed2d3902cf643)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:20.902955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0ca23-1a4d-4e7b-a34c-98f26c3a5540 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:20.905114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 134/282] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:20 2022] GET /identity/v3/users/1993525339404ca5b5aed2d3902cf643 => generated 293 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:20.905933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0ca23-1a4d-4e7b-a34c-98f26c3a5540 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:20.966955 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee6d78-a978-4691-935a-0682d731c44d None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:20.989535 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:21.018034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be68a8-9fd0-41cd-9089-ec05121d791e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:21.019252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be68a8-9fd0-41cd-9089-ec05121d791e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:21.019683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be68a8-9fd0-41cd-9089-ec05121d791e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:21.020150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be68a8-9fd0-41cd-9089-ec05121d791e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:21.035683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be68a8-9fd0-41cd-9089-ec05121d791e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-92be68a8-9fd0-41cd-9089-ec05121d791e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:21.043752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:21.044826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:21.045828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:21.045828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:21.045828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:21.045828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:21.045828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:21.045828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:21.045828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:21.045828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:21.045828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:21.045828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:21.045828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:21.045828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:21.045828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 149/283] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:21.046728 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be68a8-9fd0-41cd-9089-ec05121d791e None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:21.052549 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0ca23-1a4d-4e7b-a34c-98f26c3a5540 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:21.100351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138bf88-2f52-442c-9e2c-abaddf9177e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:21.101239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138bf88-2f52-442c-9e2c-abaddf9177e3 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:21.101747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138bf88-2f52-442c-9e2c-abaddf9177e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:21.102035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138bf88-2f52-442c-9e2c-abaddf9177e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:21.118046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138bf88-2f52-442c-9e2c-abaddf9177e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-a138bf88-2f52-442c-9e2c-abaddf9177e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:21.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:21.129899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:21.131457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:21.131457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:21.131457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:21.131457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:21.131457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:21.131457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:21.131457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:21.131457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:21.131457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:21.131457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:21.131457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:21.131457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:21.131457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 135/284] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:21.131457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138bf88-2f52-442c-9e2c-abaddf9177e3 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:21.202283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be68a8-9fd0-41cd-9089-ec05121d791e None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:21.213989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:21.256788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ad3ca-3660-4a75-b2da-11185ff43f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:21.258628 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ad3ca-3660-4a75-b2da-11185ff43f81 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:21.258923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ad3ca-3660-4a75-b2da-11185ff43f81 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:21.259497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ad3ca-3660-4a75-b2da-11185ff43f81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:21.260852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ad3ca-3660-4a75-b2da-11185ff43f81 None admin] RBAC: Adding query filter params (role.id=22f9190503114c4aba5e898e7a460546, user.id=1993525339404ca5b5aed2d3902cf643, scope.system=all) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:21.261172 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ad3ca-3660-4a75-b2da-11185ff43f81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:21.263185 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ad3ca-3660-4a75-b2da-11185ff43f81 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:21.268303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138bf88-2f52-442c-9e2c-abaddf9177e3 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:21.269737 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 150/285] 10.222.0.60 () {56 vars in 1308 bytes} [Thu Jul 21 20:38:21 2022] GET /identity/v3/role_assignments?user.id=1993525339404ca5b5aed2d3902cf643&scope.system=all&role.id=22f9190503114c4aba5e898e7a460546 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:21.270578 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ad3ca-3660-4a75-b2da-11185ff43f81 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:21.295894 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:21.317562 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ec2ab-c77f-4bff-9207-0453880fb0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:21.318468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ec2ab-c77f-4bff-9207-0453880fb0c7 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:21.318858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ec2ab-c77f-4bff-9207-0453880fb0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:21.319279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ec2ab-c77f-4bff-9207-0453880fb0c7 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:21.320407 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ec2ab-c77f-4bff-9207-0453880fb0c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:21.320859 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ec2ab-c77f-4bff-9207-0453880fb0c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:21.322458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ec2ab-c77f-4bff-9207-0453880fb0c7 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:21.333270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 136/286] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:21.339403 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ec2ab-c77f-4bff-9207-0453880fb0c7 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:21.461830 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ad3ca-3660-4a75-b2da-11185ff43f81 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:21.537108 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9dd64-8757-44d1-9fe0-d7d74a738331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:21.537867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9dd64-8757-44d1-9fe0-d7d74a738331 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:21.543736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9dd64-8757-44d1-9fe0-d7d74a738331 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:21.543736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9dd64-8757-44d1-9fe0-d7d74a738331 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:21.543736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9dd64-8757-44d1-9fe0-d7d74a738331 None admin] RBAC: Adding query filter params (name=member) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:21.543736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9dd64-8757-44d1-9fe0-d7d74a738331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:21.543736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9dd64-8757-44d1-9fe0-d7d74a738331 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:21.547977 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ec2ab-c77f-4bff-9207-0453880fb0c7 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:21.552838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 151/287] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:21.562109 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9dd64-8757-44d1-9fe0-d7d74a738331 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:21.593608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:21.648832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37cd71-f2a9-415b-8a8f-aaf41432a8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:21.649800 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37cd71-f2a9-415b-8a8f-aaf41432a8ae None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:21.650094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37cd71-f2a9-415b-8a8f-aaf41432a8ae None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:21.650377 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37cd71-f2a9-415b-8a8f-aaf41432a8ae None admin] PATH_INFO: `/v3/users/dad214f06354464091b9b8268e2b4ca0` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:21.656650 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37cd71-f2a9-415b-8a8f-aaf41432a8ae None admin] RBAC: Authorizing `identity:get_user(user_id=dad214f06354464091b9b8268e2b4ca0)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:21.658130 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37cd71-f2a9-415b-8a8f-aaf41432a8ae None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:21.660222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 137/288] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:21 2022] GET /identity/v3/users/dad214f06354464091b9b8268e2b4ca0 => generated 297 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:21.666316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37cd71-f2a9-415b-8a8f-aaf41432a8ae None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:21.696265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9dd64-8757-44d1-9fe0-d7d74a738331 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:21.760249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:21.763478 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc513d-b701-40ed-b478-f42d0f995686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:21.764495 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc513d-b701-40ed-b478-f42d0f995686 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:21.764783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc513d-b701-40ed-b478-f42d0f995686 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:21.765058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc513d-b701-40ed-b478-f42d0f995686 None admin] PATH_INFO: `/v3/users/9f1087e8c7e14a5d87c81fdc977088f1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:21.766375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc513d-b701-40ed-b478-f42d0f995686 None admin] RBAC: Authorizing `identity:get_user(user_id=9f1087e8c7e14a5d87c81fdc977088f1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:21.767912 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc513d-b701-40ed-b478-f42d0f995686 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:21.770047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 152/289] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:21 2022] GET /identity/v3/users/9f1087e8c7e14a5d87c81fdc977088f1 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:21.777278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc513d-b701-40ed-b478-f42d0f995686 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:21.830514 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37cd71-f2a9-415b-8a8f-aaf41432a8ae None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:21.877809 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7170cd6-61d9-4731-80ef-323eb3b63249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:21.879134 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7170cd6-61d9-4731-80ef-323eb3b63249 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:21.879629 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7170cd6-61d9-4731-80ef-323eb3b63249 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:21.880115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7170cd6-61d9-4731-80ef-323eb3b63249 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:21.883405 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7170cd6-61d9-4731-80ef-323eb3b63249 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:21.885170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7170cd6-61d9-4731-80ef-323eb3b63249 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:21.887514 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 138/290] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:21 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:21.893099 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7170cd6-61d9-4731-80ef-323eb3b63249 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:21.907099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:21.925786 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc513d-b701-40ed-b478-f42d0f995686 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:21.993589 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa427f9-0333-4700-8b18-8c8b36e6ee4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:21.994537 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa427f9-0333-4700-8b18-8c8b36e6ee4a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:21.994970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa427f9-0333-4700-8b18-8c8b36e6ee4a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:21.995396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa427f9-0333-4700-8b18-8c8b36e6ee4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:22.038193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7170cd6-61d9-4731-80ef-323eb3b63249 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:22.056880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:22.078457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecf0a5-34c6-4b31-b388-1d89d61efe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:22.079415 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecf0a5-34c6-4b31-b388-1d89d61efe6e None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:22.079700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecf0a5-34c6-4b31-b388-1d89d61efe6e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:22.079920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa427f9-0333-4700-8b18-8c8b36e6ee4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:22.080034 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecf0a5-34c6-4b31-b388-1d89d61efe6e None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94/users/dad214f06354464091b9b8268e2b4ca0/roles/1052f958732d4259a0658b40d1f9c193` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:22.081750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa427f9-0333-4700-8b18-8c8b36e6ee4a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:22.082112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ecf0a5-34c6-4b31-b388-1d89d61efe6e None admin] RBAC: Authorizing `identity:create_grant(project_id=3fa3d431d9cc468aafe7b895f047ff94, user_id=dad214f06354464091b9b8268e2b4ca0, role_id=1052f958732d4259a0658b40d1f9c193)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:22.083693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ecf0a5-34c6-4b31-b388-1d89d61efe6e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:22.108161 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 139/291] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:38:21 2022] PUT /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94/users/dad214f06354464091b9b8268e2b4ca0/roles/1052f958732d4259a0658b40d1f9c193 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:38:22.222385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:22.250340 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 153/292] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 20:38:22.256294 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecf0a5-34c6-4b31-b388-1d89d61efe6e None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:22.375862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecf0a5-34c6-4b31-b388-1d89d61efe6e None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:22.412881 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15617f0-da12-4ab2-8316-ea67d89c6ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:22.414398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15617f0-da12-4ab2-8316-ea67d89c6ae5 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:22.414763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15617f0-da12-4ab2-8316-ea67d89c6ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:22.414951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15617f0-da12-4ab2-8316-ea67d89c6ae5 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:22.416273 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15617f0-da12-4ab2-8316-ea67d89c6ae5 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:22.417744 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15617f0-da12-4ab2-8316-ea67d89c6ae5 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:22.419829 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 140/293] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:22 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:22.425896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa427f9-0333-4700-8b18-8c8b36e6ee4a None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:22.565951 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa427f9-0333-4700-8b18-8c8b36e6ee4a None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:22.619920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d028a4a-4558-433b-abe9-7166c579c46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:22.621033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d028a4a-4558-433b-abe9-7166c579c46d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:22.621550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d028a4a-4558-433b-abe9-7166c579c46d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:22.622026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d028a4a-4558-433b-abe9-7166c579c46d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:22.623072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d028a4a-4558-433b-abe9-7166c579c46d None admin] RBAC: Adding query filter params (role.id=22f9190503114c4aba5e898e7a460546, scope.project.id=3fa3d431d9cc468aafe7b895f047ff94, user.id=9f1087e8c7e14a5d87c81fdc977088f1) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:22.623592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d028a4a-4558-433b-abe9-7166c579c46d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:22.625288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d028a4a-4558-433b-abe9-7166c579c46d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:22.632431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 154/294] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:22 2022] GET /identity/v3/role_assignments?user.id=9f1087e8c7e14a5d87c81fdc977088f1&scope.project.id=3fa3d431d9cc468aafe7b895f047ff94&role.id=22f9190503114c4aba5e898e7a460546 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:23.309654 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7b921-f47a-4428-b2c9-39b7c47542e6 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:23.310161 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7b921-f47a-4428-b2c9-39b7c47542e6 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:23.310526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7b921-f47a-4428-b2c9-39b7c47542e6 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:23.312378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 141/295] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:23.317770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc1c79-6590-4fd5-a35e-6303472ef2a8 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:23.318359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc1c79-6590-4fd5-a35e-6303472ef2a8 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:23.318839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc1c79-6590-4fd5-a35e-6303472ef2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:23.337364 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-ffdc1c79-6590-4fd5-a35e-6303472ef2a8 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:23.338915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ffdc1c79-6590-4fd5-a35e-6303472ef2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:23.412725 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:23.432742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 155/296] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:23.456274 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7b921-f47a-4428-b2c9-39b7c47542e6 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:23.518079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:23.585716 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7b921-f47a-4428-b2c9-39b7c47542e6 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:23.642520 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328e648-4521-42fd-b55a-c2a68d85f803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:23.643733 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328e648-4521-42fd-b55a-c2a68d85f803 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:23.644160 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328e648-4521-42fd-b55a-c2a68d85f803 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:23.644656 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328e648-4521-42fd-b55a-c2a68d85f803 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:23.658513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a704a9-2d14-4aa0-b7ae-ce70991a616c None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:23.659961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a704a9-2d14-4aa0-b7ae-ce70991a616c None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:23.660743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a704a9-2d14-4aa0-b7ae-ce70991a616c None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:23.662665 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328e648-4521-42fd-b55a-c2a68d85f803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:23.664782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 156/297] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:38:23.671821 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059a297-f8bc-41c8-9643-ef5c4f4b10b0 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:23.671821 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059a297-f8bc-41c8-9643-ef5c4f4b10b0 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:23.671821 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059a297-f8bc-41c8-9643-ef5c4f4b10b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-b328e648-4521-42fd-b55a-c2a68d85f803 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:23.680418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:23.681316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:23.682079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:23.682079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:23.682079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:23.682079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:23.682079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:23.682079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:23.682079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:23.682079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:23.682079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:23.682079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:23.682079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:23.682079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:23.683130 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 142/298] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:38:23.691999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328e648-4521-42fd-b55a-c2a68d85f803 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:23.695135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-4059a297-f8bc-41c8-9643-ef5c4f4b10b0 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:23.696578 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-4059a297-f8bc-41c8-9643-ef5c4f4b10b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:23.798310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 157/299] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:23.824922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059a297-f8bc-41c8-9643-ef5c4f4b10b0 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:23.847175 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328e648-4521-42fd-b55a-c2a68d85f803 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:23.852097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:23.894931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba800457-5183-48c2-bf28-8bd7fe1091a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:23.896282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba800457-5183-48c2-bf28-8bd7fe1091a9 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:23.896953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba800457-5183-48c2-bf28-8bd7fe1091a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:23.897518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba800457-5183-48c2-bf28-8bd7fe1091a9 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:23.899504 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba800457-5183-48c2-bf28-8bd7fe1091a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:23.900065 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba800457-5183-48c2-bf28-8bd7fe1091a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:23.903148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba800457-5183-48c2-bf28-8bd7fe1091a9 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:23.916812 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 143/300] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:23.923090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba800457-5183-48c2-bf28-8bd7fe1091a9 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:23.963020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059a297-f8bc-41c8-9643-ef5c4f4b10b0 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:23.977045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:24.011455 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7172021-5a05-45ed-8b73-736583a4d229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:24.012705 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7172021-5a05-45ed-8b73-736583a4d229 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:24.012992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7172021-5a05-45ed-8b73-736583a4d229 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:24.013270 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7172021-5a05-45ed-8b73-736583a4d229 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:24.014771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7172021-5a05-45ed-8b73-736583a4d229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:24.017035 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7172021-5a05-45ed-8b73-736583a4d229 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:24.019882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 158/301] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:24.027618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7172021-5a05-45ed-8b73-736583a4d229 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:24.074305 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba800457-5183-48c2-bf28-8bd7fe1091a9 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:24.120283 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6c3b8-1b17-49f1-a488-96a2f3fab19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:24.121510 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6c3b8-1b17-49f1-a488-96a2f3fab19d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:24.121796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6c3b8-1b17-49f1-a488-96a2f3fab19d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:24.122072 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6c3b8-1b17-49f1-a488-96a2f3fab19d None admin] PATH_INFO: `/v3/users/61fafebcd45c4520bc4c2c8aa4fac9d7` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:24.138202 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6c3b8-1b17-49f1-a488-96a2f3fab19d None admin] RBAC: Authorizing `identity:get_user(user_id=61fafebcd45c4520bc4c2c8aa4fac9d7)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:24.139671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6c3b8-1b17-49f1-a488-96a2f3fab19d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:24.141476 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 144/302] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:23 2022] GET /identity/v3/users/61fafebcd45c4520bc4c2c8aa4fac9d7 => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:24.142769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11201c14-7e28-4cd4-b85d-be4e699c8d3a None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:24.143051 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11201c14-7e28-4cd4-b85d-be4e699c8d3a None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:24.143323 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11201c14-7e28-4cd4-b85d-be4e699c8d3a None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:24.147857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 145/303] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:24.148303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11201c14-7e28-4cd4-b85d-be4e699c8d3a None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:24.156379 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7172021-5a05-45ed-8b73-736583a4d229 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:24.178463 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:24.199289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c079c9-87a8-4d61-b638-41e2bcada8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:24.200012 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c079c9-87a8-4d61-b638-41e2bcada8b5 None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:24.200292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c079c9-87a8-4d61-b638-41e2bcada8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:24.200572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c079c9-87a8-4d61-b638-41e2bcada8b5 None admin] PATH_INFO: `/v3/groups` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:24.201276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c079c9-87a8-4d61-b638-41e2bcada8b5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:24.202744 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c079c9-87a8-4d61-b638-41e2bcada8b5 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:24.219306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 159/304] 10.222.0.60 () {60 vars in 1134 bytes} [Thu Jul 21 20:38:24 2022] POST /identity/v3/groups => generated 229 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:38:24.221546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3aefb-202b-44bb-8d6a-82a25104cb9c None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:24.222047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3aefb-202b-44bb-8d6a-82a25104cb9c None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:24.222670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3aefb-202b-44bb-8d6a-82a25104cb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:24.243546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-06e3aefb-202b-44bb-8d6a-82a25104cb9c None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:24.244886 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-06e3aefb-202b-44bb-8d6a-82a25104cb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:24.289547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:24.306881 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11201c14-7e28-4cd4-b85d-be4e699c8d3a None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:24.355828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9dd65-e3bd-41da-b2a2-88d4f9899bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:24.356488 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 160/305] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:24.357326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9dd65-e3bd-41da-b2a2-88d4f9899bbe None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:24.357829 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9dd65-e3bd-41da-b2a2-88d4f9899bbe None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:24.358240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9dd65-e3bd-41da-b2a2-88d4f9899bbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:24.359296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9dd65-e3bd-41da-b2a2-88d4f9899bbe None admin] RBAC: Adding query filter params (role.id=1052f958732d4259a0658b40d1f9c193, user.id=61fafebcd45c4520bc4c2c8aa4fac9d7, scope.system=all) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:24.359781 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9dd65-e3bd-41da-b2a2-88d4f9899bbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:24.361549 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9dd65-e3bd-41da-b2a2-88d4f9899bbe None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:24.370603 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 146/306] 10.222.0.60 () {56 vars in 1308 bytes} [Thu Jul 21 20:38:24 2022] GET /identity/v3/role_assignments?user.id=61fafebcd45c4520bc4c2c8aa4fac9d7&scope.system=all&role.id=1052f958732d4259a0658b40d1f9c193 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:24.375418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3aefb-202b-44bb-8d6a-82a25104cb9c None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:24.510082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3aefb-202b-44bb-8d6a-82a25104cb9c None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:24.524164 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9be60-15f4-4601-b285-0fe14dcffda6 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:24.524625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9be60-15f4-4601-b285-0fe14dcffda6 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:24.524933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9be60-15f4-4601-b285-0fe14dcffda6 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:24.526669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 147/307] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:24.533953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b4f71-410a-468e-83dc-0d72f0202803 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:24.534263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b4f71-410a-468e-83dc-0d72f0202803 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:24.534515 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b4f71-410a-468e-83dc-0d72f0202803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:24.555004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde2ba3-6be1-42ec-b39a-0639383ee5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:24.556224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde2ba3-6be1-42ec-b39a-0639383ee5e5 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:24.556707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde2ba3-6be1-42ec-b39a-0639383ee5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:24.557197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde2ba3-6be1-42ec-b39a-0639383ee5e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:24.558506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-bf2b4f71-410a-468e-83dc-0d72f0202803 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:24.562059 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-bf2b4f71-410a-468e-83dc-0d72f0202803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:24.578508 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde2ba3-6be1-42ec-b39a-0639383ee5e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-8cde2ba3-6be1-42ec-b39a-0639383ee5e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:24.588634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:24.590311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:24.591655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:24.591655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:24.591655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:24.591655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:24.591655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:24.591655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:24.591655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:24.591655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:24.591655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:24.591655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:24.591655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:24.591655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:24.594671 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 161/308] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:24.600222 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde2ba3-6be1-42ec-b39a-0639383ee5e5 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:24.644370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:24.649997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 148/309] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:24.673605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b4f71-410a-468e-83dc-0d72f0202803 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:24.701757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:24.734102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde2ba3-6be1-42ec-b39a-0639383ee5e5 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:24.777087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b443d-aaab-49fe-bba3-4799c9be4da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:24.777773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b443d-aaab-49fe-bba3-4799c9be4da5 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:24.778057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b443d-aaab-49fe-bba3-4799c9be4da5 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:24.778334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b443d-aaab-49fe-bba3-4799c9be4da5 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:24.779100 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b443d-aaab-49fe-bba3-4799c9be4da5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:24.779408 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b443d-aaab-49fe-bba3-4799c9be4da5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:24.780847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b443d-aaab-49fe-bba3-4799c9be4da5 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:24.791104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 162/310] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:24.796579 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b443d-aaab-49fe-bba3-4799c9be4da5 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:24.816288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b4f71-410a-468e-83dc-0d72f0202803 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:24.856665 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87433a1c-7791-4b73-9228-03648cebb2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:24.857871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87433a1c-7791-4b73-9228-03648cebb2cd None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:24.858396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87433a1c-7791-4b73-9228-03648cebb2cd None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:24.858848 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87433a1c-7791-4b73-9228-03648cebb2cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:24.871203 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87433a1c-7791-4b73-9228-03648cebb2cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-87433a1c-7791-4b73-9228-03648cebb2cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:24.879928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:24.880803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:24.881555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:24.881555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:24.881555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:24.881555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:24.881555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:24.881555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:24.881555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:24.881555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:24.881555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:24.881555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:24.881555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:24.881555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:24.889013 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 149/311] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:38:24.890643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87433a1c-7791-4b73-9228-03648cebb2cd None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:24.906076 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:24.919150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b443d-aaab-49fe-bba3-4799c9be4da5 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:24.961884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc7db02-cd6a-418c-841d-f8d966d3e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:24.963530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7db02-cd6a-418c-841d-f8d966d3e21d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:24.963971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7db02-cd6a-418c-841d-f8d966d3e21d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:24.964312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7db02-cd6a-418c-841d-f8d966d3e21d None admin] PATH_INFO: `/v3/users/1993525339404ca5b5aed2d3902cf643` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:24.965768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc7db02-cd6a-418c-841d-f8d966d3e21d None admin] RBAC: Authorizing `identity:get_user(user_id=1993525339404ca5b5aed2d3902cf643)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:24.967248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc7db02-cd6a-418c-841d-f8d966d3e21d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:24.969355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 163/312] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:24 2022] GET /identity/v3/users/1993525339404ca5b5aed2d3902cf643 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:24.977731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc7db02-cd6a-418c-841d-f8d966d3e21d None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:24.985087 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:25.009671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87433a1c-7791-4b73-9228-03648cebb2cd None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:25.048272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa9904-91e5-4658-ba4b-172afc182749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:25.049308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa9904-91e5-4658-ba4b-172afc182749 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:25.049834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa9904-91e5-4658-ba4b-172afc182749 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:25.050582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa9904-91e5-4658-ba4b-172afc182749 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:25.051658 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa9904-91e5-4658-ba4b-172afc182749 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:25.052196 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa9904-91e5-4658-ba4b-172afc182749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:25.054318 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa9904-91e5-4658-ba4b-172afc182749 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:25.064607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 150/313] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:25.071635 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa9904-91e5-4658-ba4b-172afc182749 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:25.111588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc7db02-cd6a-418c-841d-f8d966d3e21d None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:25.159834 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ec3b2-be38-49cb-9484-97151f5d2001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:25.160870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ec3b2-be38-49cb-9484-97151f5d2001 None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:25.161168 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ec3b2-be38-49cb-9484-97151f5d2001 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:25.161449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ec3b2-be38-49cb-9484-97151f5d2001 None admin] PATH_INFO: `/v3/system/users/1993525339404ca5b5aed2d3902cf643/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:25.164214 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ec3b2-be38-49cb-9484-97151f5d2001 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1993525339404ca5b5aed2d3902cf643, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:25.165756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ec3b2-be38-49cb-9484-97151f5d2001 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:25.176401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 164/314] 10.222.0.60 () {58 vars in 1334 bytes} [Thu Jul 21 20:38:24 2022] PUT /identity/v3/system/users/1993525339404ca5b5aed2d3902cf643/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:38:25.196795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa9904-91e5-4658-ba4b-172afc182749 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:25.244240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b571022-20e4-4cce-b485-119fdb9f70fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:25.245742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b571022-20e4-4cce-b485-119fdb9f70fb None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:25.246324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b571022-20e4-4cce-b485-119fdb9f70fb None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:25.246860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b571022-20e4-4cce-b485-119fdb9f70fb None admin] PATH_INFO: `/v3/users/dad214f06354464091b9b8268e2b4ca0` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:25.248617 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b571022-20e4-4cce-b485-119fdb9f70fb None admin] RBAC: Authorizing `identity:get_user(user_id=dad214f06354464091b9b8268e2b4ca0)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:25.250501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b571022-20e4-4cce-b485-119fdb9f70fb None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:25.257528 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 151/315] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:25 2022] GET /identity/v3/users/dad214f06354464091b9b8268e2b4ca0 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:25.259339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ec3b2-be38-49cb-9484-97151f5d2001 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:25.286823 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:25.384600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ec3b2-be38-49cb-9484-97151f5d2001 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:25.423057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e8a94-4a4a-42ff-b0a0-6f30d81f1593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:25.423637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e8a94-4a4a-42ff-b0a0-6f30d81f1593 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:25.423988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e8a94-4a4a-42ff-b0a0-6f30d81f1593 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:25.424262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e8a94-4a4a-42ff-b0a0-6f30d81f1593 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:25.500221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e8a94-4a4a-42ff-b0a0-6f30d81f1593 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:25.501809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e8a94-4a4a-42ff-b0a0-6f30d81f1593 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:25.525004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:25.623584 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 165/316] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:25.625344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b571022-20e4-4cce-b485-119fdb9f70fb None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:25.639288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:25.738458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b571022-20e4-4cce-b485-119fdb9f70fb None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:25.778989 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54d52a-c811-4415-9f5d-eb7c69e2ee5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:25.780248 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54d52a-c811-4415-9f5d-eb7c69e2ee5b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:25.780727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54d52a-c811-4415-9f5d-eb7c69e2ee5b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:25.781190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54d52a-c811-4415-9f5d-eb7c69e2ee5b None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:25.782599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54d52a-c811-4415-9f5d-eb7c69e2ee5b None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:25.784307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54d52a-c811-4415-9f5d-eb7c69e2ee5b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:25.786259 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 152/317] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:25 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:25.792833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e8a94-4a4a-42ff-b0a0-6f30d81f1593 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:25.919348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e8a94-4a4a-42ff-b0a0-6f30d81f1593 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:25.943624 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:25.961790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b149a2-3230-49a6-b75c-9a26e5273704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:25.963169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b149a2-3230-49a6-b75c-9a26e5273704 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:25.964262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b149a2-3230-49a6-b75c-9a26e5273704 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:25.964798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b149a2-3230-49a6-b75c-9a26e5273704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:25.965808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b149a2-3230-49a6-b75c-9a26e5273704 None admin] RBAC: Adding query filter params (role.id=1052f958732d4259a0658b40d1f9c193, scope.project.id=3fa3d431d9cc468aafe7b895f047ff94, user.id=dad214f06354464091b9b8268e2b4ca0) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:25.966306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b149a2-3230-49a6-b75c-9a26e5273704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:25.967961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b149a2-3230-49a6-b75c-9a26e5273704 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:25.975897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 166/318] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:25 2022] GET /identity/v3/role_assignments?user.id=dad214f06354464091b9b8268e2b4ca0&scope.project.id=3fa3d431d9cc468aafe7b895f047ff94&role.id=1052f958732d4259a0658b40d1f9c193 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:26.554232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb52529-5745-4f64-925f-f90e47efa749 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:26.554884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb52529-5745-4f64-925f-f90e47efa749 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:26.555246 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb52529-5745-4f64-925f-f90e47efa749 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:26.556715 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 153/319] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:26.562367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf55d4-93cd-499e-9d58-05dbc8f8e597 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:26.562638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf55d4-93cd-499e-9d58-05dbc8f8e597 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:26.562970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf55d4-93cd-499e-9d58-05dbc8f8e597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:26.581003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-bdbf55d4-93cd-499e-9d58-05dbc8f8e597 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:26.582201 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-bdbf55d4-93cd-499e-9d58-05dbc8f8e597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:26.611940 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca61178-1b11-4e56-afb7-07a5990dde5f None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:26.612227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca61178-1b11-4e56-afb7-07a5990dde5f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:26.612508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca61178-1b11-4e56-afb7-07a5990dde5f None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:26.615618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 154/320] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:26.623523 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46baedca-cf60-48fe-96a7-5869313419dc None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:26.623834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46baedca-cf60-48fe-96a7-5869313419dc None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:26.624139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46baedca-cf60-48fe-96a7-5869313419dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:26.650619 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-46baedca-cf60-48fe-96a7-5869313419dc None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:26.652029 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-46baedca-cf60-48fe-96a7-5869313419dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:26.673154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 167/321] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:26.697324 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf55d4-93cd-499e-9d58-05dbc8f8e597 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:26.743993 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 155/322] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:26.768131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46baedca-cf60-48fe-96a7-5869313419dc None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:26.782367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:26.822502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf55d4-93cd-499e-9d58-05dbc8f8e597 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:26.841436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:26.865637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfa65d-89fb-4671-a4de-401fd935d19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:26.866801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfa65d-89fb-4671-a4de-401fd935d19c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:26.867287 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfa65d-89fb-4671-a4de-401fd935d19c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:26.867615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfa65d-89fb-4671-a4de-401fd935d19c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:26.868917 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfa65d-89fb-4671-a4de-401fd935d19c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:26.870381 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfa65d-89fb-4671-a4de-401fd935d19c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:26.872089 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 168/323] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:26.877731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfa65d-89fb-4671-a4de-401fd935d19c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:26.901148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46baedca-cf60-48fe-96a7-5869313419dc None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:26.940905 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8aa265-043d-4c3b-bc3d-46fa9acf6b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:26.941773 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aa265-043d-4c3b-bc3d-46fa9acf6b0a None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:26.942062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aa265-043d-4c3b-bc3d-46fa9acf6b0a None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:26.942338 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aa265-043d-4c3b-bc3d-46fa9acf6b0a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:26.956757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8aa265-043d-4c3b-bc3d-46fa9acf6b0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-2e8aa265-043d-4c3b-bc3d-46fa9acf6b0a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:26.964506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:26.965466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:26.966227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:26.966227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:26.966227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:26.966227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:26.966227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:26.966227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:26.966227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:26.966227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:26.966227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:26.966227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:26.966227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:26.966227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:26.966227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 156/324] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:38:26.971499 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8aa265-043d-4c3b-bc3d-46fa9acf6b0a None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:26.999467 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfa65d-89fb-4671-a4de-401fd935d19c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:27.044321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e757a-4efa-4a2a-9fab-71e2250d8860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:27.044996 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e757a-4efa-4a2a-9fab-71e2250d8860 None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:27.045277 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e757a-4efa-4a2a-9fab-71e2250d8860 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:27.045553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e757a-4efa-4a2a-9fab-71e2250d8860 None admin] PATH_INFO: `/v3/groups` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:27.046345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e757a-4efa-4a2a-9fab-71e2250d8860 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:27.047398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:27.048665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e757a-4efa-4a2a-9fab-71e2250d8860 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:27.072049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 169/325] 10.222.0.60 () {60 vars in 1134 bytes} [Thu Jul 21 20:38:26 2022] POST /identity/v3/groups => generated 226 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:38:27.103141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8aa265-043d-4c3b-bc3d-46fa9acf6b0a None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:27.154769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfea4b-4b58-433e-b0b7-c8913ceb0e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:27.156032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfea4b-4b58-433e-b0b7-c8913ceb0e3c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:27.156539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfea4b-4b58-433e-b0b7-c8913ceb0e3c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:27.157010 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfea4b-4b58-433e-b0b7-c8913ceb0e3c None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:27.158006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfea4b-4b58-433e-b0b7-c8913ceb0e3c None admin] RBAC: Adding query filter params (name=reader) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:27.158752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfea4b-4b58-433e-b0b7-c8913ceb0e3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:27.160579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfea4b-4b58-433e-b0b7-c8913ceb0e3c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:27.171021 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 157/326] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:27.177063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e757a-4efa-4a2a-9fab-71e2250d8860 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:27.235150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:27.307487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e757a-4efa-4a2a-9fab-71e2250d8860 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:27.344343 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00c8e8-083c-4106-a576-7bb2cb06bd7b None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:27.344633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00c8e8-083c-4106-a576-7bb2cb06bd7b None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:27.344910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00c8e8-083c-4106-a576-7bb2cb06bd7b None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:27.346283 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 158/327] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:27.347133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d80b49-ec02-4546-aeb3-2523b0deac1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:27.348057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d80b49-ec02-4546-aeb3-2523b0deac1c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:27.348346 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d80b49-ec02-4546-aeb3-2523b0deac1c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:27.348629 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d80b49-ec02-4546-aeb3-2523b0deac1c None admin] PATH_INFO: `/v3/users/61fafebcd45c4520bc4c2c8aa4fac9d7` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:27.349879 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d80b49-ec02-4546-aeb3-2523b0deac1c None admin] RBAC: Authorizing `identity:get_user(user_id=61fafebcd45c4520bc4c2c8aa4fac9d7)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:27.351651 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d80b49-ec02-4546-aeb3-2523b0deac1c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:27.351818 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3546989-bed0-4e99-90a6-630759bee0d7 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:27.351989 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3546989-bed0-4e99-90a6-630759bee0d7 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:27.352254 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3546989-bed0-4e99-90a6-630759bee0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:27.353531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 170/328] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:27 2022] GET /identity/v3/users/61fafebcd45c4520bc4c2c8aa4fac9d7 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:27.360063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d80b49-ec02-4546-aeb3-2523b0deac1c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:27.375187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-a3546989-bed0-4e99-90a6-630759bee0d7 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:27.376444 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a3546989-bed0-4e99-90a6-630759bee0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:27.470223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 159/329] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:38:27.494592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d80b49-ec02-4546-aeb3-2523b0deac1c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:27.494825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3546989-bed0-4e99-90a6-630759bee0d7 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:27.508533 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:27.549965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7a7ef-46e7-4ddd-a985-35e90649fc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:27.550852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7a7ef-46e7-4ddd-a985-35e90649fc99 None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:27.551145 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7a7ef-46e7-4ddd-a985-35e90649fc99 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:27.551429 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7a7ef-46e7-4ddd-a985-35e90649fc99 None admin] PATH_INFO: `/v3/system/users/61fafebcd45c4520bc4c2c8aa4fac9d7/roles/1052f958732d4259a0658b40d1f9c193` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:27.555492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7a7ef-46e7-4ddd-a985-35e90649fc99 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=61fafebcd45c4520bc4c2c8aa4fac9d7, role_id=1052f958732d4259a0658b40d1f9c193)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:27.556963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7a7ef-46e7-4ddd-a985-35e90649fc99 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:27.567189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 171/330] 10.222.0.60 () {58 vars in 1334 bytes} [Thu Jul 21 20:38:27 2022] PUT /identity/v3/system/users/61fafebcd45c4520bc4c2c8aa4fac9d7/roles/1052f958732d4259a0658b40d1f9c193 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:38:27.576796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:27.638180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3546989-bed0-4e99-90a6-630759bee0d7 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:27.689601 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f17a31-e3dd-4993-bfe3-387814845bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:27.691262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f17a31-e3dd-4993-bfe3-387814845bf1 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:27.691852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f17a31-e3dd-4993-bfe3-387814845bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:27.692337 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f17a31-e3dd-4993-bfe3-387814845bf1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:27.705460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f17a31-e3dd-4993-bfe3-387814845bf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-36f17a31-e3dd-4993-bfe3-387814845bf1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:27.714536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:27.716159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:27.717727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:27.717727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:27.717727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:27.717727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:27.717727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:27.717727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:27.717727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:27.717727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:27.717727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:27.717727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:27.717727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:27.717727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:27.721046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 160/331] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:38:27.728896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7a7ef-46e7-4ddd-a985-35e90649fc99 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:27.860858 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7a7ef-46e7-4ddd-a985-35e90649fc99 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:27.909561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d5e50-ff5a-4fa8-86fd-94abfd97948c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:27.910690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d5e50-ff5a-4fa8-86fd-94abfd97948c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:27.911152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d5e50-ff5a-4fa8-86fd-94abfd97948c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:27.911630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d5e50-ff5a-4fa8-86fd-94abfd97948c None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:27.913767 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d5e50-ff5a-4fa8-86fd-94abfd97948c None admin] RBAC: Adding query filter params (name=member) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:27.914330 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d5e50-ff5a-4fa8-86fd-94abfd97948c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:27.916837 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d5e50-ff5a-4fa8-86fd-94abfd97948c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:27.929785 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 172/332] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:27.936108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f17a31-e3dd-4993-bfe3-387814845bf1 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:28.087762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f17a31-e3dd-4993-bfe3-387814845bf1 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:28.112043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:28.139514 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac946234-35a3-49d7-8cc3-18128cb9db7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:28.140460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac946234-35a3-49d7-8cc3-18128cb9db7a None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:28.140741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac946234-35a3-49d7-8cc3-18128cb9db7a None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:28.141014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac946234-35a3-49d7-8cc3-18128cb9db7a None admin] PATH_INFO: `/v3/users/1993525339404ca5b5aed2d3902cf643` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:28.142239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac946234-35a3-49d7-8cc3-18128cb9db7a None admin] RBAC: Authorizing `identity:get_user(user_id=1993525339404ca5b5aed2d3902cf643)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:28.143729 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac946234-35a3-49d7-8cc3-18128cb9db7a None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:28.145694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 161/333] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:27 2022] GET /identity/v3/users/1993525339404ca5b5aed2d3902cf643 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:28.153459 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d5e50-ff5a-4fa8-86fd-94abfd97948c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:28.225004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:28.310259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d5e50-ff5a-4fa8-86fd-94abfd97948c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:28.357465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86592cbd-1dbc-41a7-8101-ef287a20e032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:28.358132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86592cbd-1dbc-41a7-8101-ef287a20e032 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:28.358412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86592cbd-1dbc-41a7-8101-ef287a20e032 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:28.358693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86592cbd-1dbc-41a7-8101-ef287a20e032 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:28.359526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86592cbd-1dbc-41a7-8101-ef287a20e032 None admin] RBAC: Adding query filter params (role.id=22f9190503114c4aba5e898e7a460546, user.id=1993525339404ca5b5aed2d3902cf643, scope.system=all) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:28.359874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86592cbd-1dbc-41a7-8101-ef287a20e032 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:28.361311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86592cbd-1dbc-41a7-8101-ef287a20e032 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:28.375069 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 173/334] 10.222.0.60 () {56 vars in 1308 bytes} [Thu Jul 21 20:38:28 2022] GET /identity/v3/role_assignments?user.id=1993525339404ca5b5aed2d3902cf643&scope.system=all&role.id=22f9190503114c4aba5e898e7a460546 => generated 510 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:29.773668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfdfb9-4a78-4393-a287-3b42a55cf2db None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:29.774417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfdfb9-4a78-4393-a287-3b42a55cf2db None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:29.774417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfdfb9-4a78-4393-a287-3b42a55cf2db None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:29.777157 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 162/335] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:29.783635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc51a2-2cf2-479d-ab20-1437db010f2b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:29.784199 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc51a2-2cf2-479d-ab20-1437db010f2b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:29.784282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc51a2-2cf2-479d-ab20-1437db010f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:29.813860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-4ffc51a2-2cf2-479d-ab20-1437db010f2b None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:29.815131 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-4ffc51a2-2cf2-479d-ab20-1437db010f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:29.927978 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 174/336] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:29.978822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfdfb9-4a78-4393-a287-3b42a55cf2db None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:30.151249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfdfb9-4a78-4393-a287-3b42a55cf2db None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:30.205574 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4f317-3c06-45c4-a77e-d69046f0cc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:30.206617 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4f317-3c06-45c4-a77e-d69046f0cc70 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:30.206809 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4f317-3c06-45c4-a77e-d69046f0cc70 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:30.207028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4f317-3c06-45c4-a77e-d69046f0cc70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:30.225117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4f317-3c06-45c4-a77e-d69046f0cc70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-83f4f317-3c06-45c4-a77e-d69046f0cc70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:30.234340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:30.235426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:30.236310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:30.236310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:30.236310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:30.236310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:30.236310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:30.236310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:30.236310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:30.236310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:30.236310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:30.236310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:30.236310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:30.236310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:30.238586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 163/337] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:30.247106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc51a2-2cf2-479d-ab20-1437db010f2b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:30.296477 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:30.420807 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc51a2-2cf2-479d-ab20-1437db010f2b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:30.467332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b662c-65c8-4c7a-9621-6f3899373b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:30.468067 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b662c-65c8-4c7a-9621-6f3899373b02 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:30.468356 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b662c-65c8-4c7a-9621-6f3899373b02 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:30.468635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b662c-65c8-4c7a-9621-6f3899373b02 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:30.469430 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b662c-65c8-4c7a-9621-6f3899373b02 None admin] RBAC: Adding query filter params (name=member) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:30.469742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b662c-65c8-4c7a-9621-6f3899373b02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:30.471213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b662c-65c8-4c7a-9621-6f3899373b02 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:30.482235 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 175/338] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:30.488322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4f317-3c06-45c4-a77e-d69046f0cc70 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:30.511484 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34b02a-cc90-4918-991f-cb04d47e19b6 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:30.511926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34b02a-cc90-4918-991f-cb04d47e19b6 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:30.512214 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34b02a-cc90-4918-991f-cb04d47e19b6 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:30.513869 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 176/339] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:30.519761 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0aabe-623c-47ab-ab34-f052e1f8ddcc None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:30.520098 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0aabe-623c-47ab-ab34-f052e1f8ddcc None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:30.520423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0aabe-623c-47ab-ab34-f052e1f8ddcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:30.552856 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-52b0aabe-623c-47ab-ab34-f052e1f8ddcc None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:30.554137 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-52b0aabe-623c-47ab-ab34-f052e1f8ddcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:30.608674 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:30.724006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4f317-3c06-45c4-a77e-d69046f0cc70 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:30.726012 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 177/340] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:30.797917 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79ec13-fef8-4356-b992-6e1ee780b578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:30.798468 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0aabe-623c-47ab-ab34-f052e1f8ddcc None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:30.798814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79ec13-fef8-4356-b992-6e1ee780b578 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:30.798892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79ec13-fef8-4356-b992-6e1ee780b578 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:30.799144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79ec13-fef8-4356-b992-6e1ee780b578 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:30.852115 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:30.918793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79ec13-fef8-4356-b992-6e1ee780b578 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:30.920791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79ec13-fef8-4356-b992-6e1ee780b578 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:30.953225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0aabe-623c-47ab-ab34-f052e1f8ddcc None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:30.996815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5839cb-3a86-46ae-8ef5-db1ca8adb769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:30.999824 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5839cb-3a86-46ae-8ef5-db1ca8adb769 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:31.003430 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5839cb-3a86-46ae-8ef5-db1ca8adb769 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:31.004206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5839cb-3a86-46ae-8ef5-db1ca8adb769 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:31.008152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:31.030977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5839cb-3a86-46ae-8ef5-db1ca8adb769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-dc5839cb-3a86-46ae-8ef5-db1ca8adb769 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:31.048345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:31.049292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:31.050490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:31.050490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:31.050490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:31.050490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:31.050490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:31.050490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:31.050490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:31.050490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:31.050490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:31.050490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:31.050490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:31.050490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:31.050490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 178/341] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:38:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:38:31.055452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfab4cb-c549-4e94-9159-6d8fe0aa71b1 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:31.057126 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfab4cb-c549-4e94-9159-6d8fe0aa71b1 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:31.058946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfab4cb-c549-4e94-9159-6d8fe0aa71b1 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:31.060705 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 179/342] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:31.064211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfab4cb-c549-4e94-9159-6d8fe0aa71b1 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:31.126405 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 164/343] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:31.127792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb973e5-2159-4588-bd0e-1ae378195309 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:31.128088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb973e5-2159-4588-bd0e-1ae378195309 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:31.128388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb973e5-2159-4588-bd0e-1ae378195309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:31.167052 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-1cb973e5-2159-4588-bd0e-1ae378195309 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:31.168303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1cb973e5-2159-4588-bd0e-1ae378195309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:31.222412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:31.270953 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfab4cb-c549-4e94-9159-6d8fe0aa71b1 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:31.285306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 165/344] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:31.286809 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb973e5-2159-4588-bd0e-1ae378195309 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:31.316792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a2467-5991-4ee9-aabc-b0a1e7da9533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:31.318131 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a2467-5991-4ee9-aabc-b0a1e7da9533 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:31.319784 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a2467-5991-4ee9-aabc-b0a1e7da9533 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:31.320603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a2467-5991-4ee9-aabc-b0a1e7da9533 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:31.321675 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a2467-5991-4ee9-aabc-b0a1e7da9533 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:31.322409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a2467-5991-4ee9-aabc-b0a1e7da9533 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:31.324831 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a2467-5991-4ee9-aabc-b0a1e7da9533 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:31.337990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 180/345] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:31.339679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ea77d-e45c-40d7-9f90-3473f0176933 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:31.340159 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ea77d-e45c-40d7-9f90-3473f0176933 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:31.340589 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ea77d-e45c-40d7-9f90-3473f0176933 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:31.342242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 181/346] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:31.343237 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ea77d-e45c-40d7-9f90-3473f0176933 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:31.399232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:31.446588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb973e5-2159-4588-bd0e-1ae378195309 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:31.494590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030b35a-c826-47c4-9484-df4d38acf733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:31.495481 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030b35a-c826-47c4-9484-df4d38acf733 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:31.495788 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030b35a-c826-47c4-9484-df4d38acf733 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:31.496251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030b35a-c826-47c4-9484-df4d38acf733 None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:31.496439 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ea77d-e45c-40d7-9f90-3473f0176933 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:31.497667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030b35a-c826-47c4-9484-df4d38acf733 None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:31.499123 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030b35a-c826-47c4-9484-df4d38acf733 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:31.501232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 166/347] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:31 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:31.502180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030b35a-c826-47c4-9484-df4d38acf733 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:31.550524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaee5dc-e14b-4161-a5fd-765643c3a01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:31.551497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaee5dc-e14b-4161-a5fd-765643c3a01e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:31.551770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaee5dc-e14b-4161-a5fd-765643c3a01e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:31.552055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaee5dc-e14b-4161-a5fd-765643c3a01e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:31.564937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:31.571807 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaee5dc-e14b-4161-a5fd-765643c3a01e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-feaee5dc-e14b-4161-a5fd-765643c3a01e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:31.580659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:31.581679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:31.582458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:31.582458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:31.582458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:31.582458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:31.582458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:31.582458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:31.582458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:31.582458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:31.582458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:31.582458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:31.582458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 20:38:31.582458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:31.585800 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 182/348] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:31.587423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32aa60-d057-4260-98c6-20b1a5dcc42c None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:31.587913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32aa60-d057-4260-98c6-20b1a5dcc42c None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:31.588376 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32aa60-d057-4260-98c6-20b1a5dcc42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:31.608035 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-9c32aa60-d057-4260-98c6-20b1a5dcc42c None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:31.609639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9c32aa60-d057-4260-98c6-20b1a5dcc42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:31.648834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030b35a-c826-47c4-9484-df4d38acf733 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:31.694304 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 183/349] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:31.695932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32aa60-d057-4260-98c6-20b1a5dcc42c None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:31.706507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e8376-ffb6-4167-ab28-209f68e0e662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:31.707182 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e8376-ffb6-4167-ab28-209f68e0e662 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:31.707468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e8376-ffb6-4167-ab28-209f68e0e662 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:31.707761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e8376-ffb6-4167-ab28-209f68e0e662 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:31.742181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:31.811096 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e8376-ffb6-4167-ab28-209f68e0e662 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:31.812659 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e8376-ffb6-4167-ab28-209f68e0e662 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:31.852310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32aa60-d057-4260-98c6-20b1a5dcc42c None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:31.903423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:31.916604 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b484063-3527-4a60-b327-93bda589cf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:31.917486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b484063-3527-4a60-b327-93bda589cf88 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:31.917769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b484063-3527-4a60-b327-93bda589cf88 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:31.918051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b484063-3527-4a60-b327-93bda589cf88 None admin] PATH_INFO: `/v3/groups/895fc55542e14dbbaa927216b4e9442f` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:31.926258 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b484063-3527-4a60-b327-93bda589cf88 None admin] RBAC: Authorizing `identity:get_group(group_id=895fc55542e14dbbaa927216b4e9442f)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:31.928409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b484063-3527-4a60-b327-93bda589cf88 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:31.930476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 184/350] 10.222.0.60 () {56 vars in 1180 bytes} [Thu Jul 21 20:38:31 2022] GET /identity/v3/groups/895fc55542e14dbbaa927216b4e9442f => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:31.936532 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b484063-3527-4a60-b327-93bda589cf88 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:31.955768 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 167/351] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:31.961696 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e8376-ffb6-4167-ab28-209f68e0e662 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:32.034145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:32.106043 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b484063-3527-4a60-b327-93bda589cf88 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:32.123308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e8376-ffb6-4167-ab28-209f68e0e662 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:32.157411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e454a8-8155-4997-bfe9-f56f19ae79cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:32.158096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e454a8-8155-4997-bfe9-f56f19ae79cf None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:32.158383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e454a8-8155-4997-bfe9-f56f19ae79cf None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:32.158659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e454a8-8155-4997-bfe9-f56f19ae79cf None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:32.159425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e454a8-8155-4997-bfe9-f56f19ae79cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:32.160439 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e454a8-8155-4997-bfe9-f56f19ae79cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:32.161869 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e454a8-8155-4997-bfe9-f56f19ae79cf None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:32.168912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8e655-c7c1-438a-b6cc-7e04dd99101f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:32.169125 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e655-c7c1-438a-b6cc-7e04dd99101f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:32.169373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e655-c7c1-438a-b6cc-7e04dd99101f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:32.169659 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e655-c7c1-438a-b6cc-7e04dd99101f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:32.174345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 185/352] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:32.175909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e745d-cd38-4791-b1d7-28f12321cfe4 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:32.175909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e745d-cd38-4791-b1d7-28f12321cfe4 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:32.176327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e745d-cd38-4791-b1d7-28f12321cfe4 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:32.177679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 186/353] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:32.179075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e745d-cd38-4791-b1d7-28f12321cfe4 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:32.193359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8e655-c7c1-438a-b6cc-7e04dd99101f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-3fa8e655-c7c1-438a-b6cc-7e04dd99101f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:32.203410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:32.204465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:32.205378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:32.205378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:32.205378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:32.205378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:32.205378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:32.205378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:32.205378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:32.205378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:32.205378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:32.205378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:32.205378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:32.205378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:32.205378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 168/354] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:32.206147 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8e655-c7c1-438a-b6cc-7e04dd99101f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:32.254976 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:32.329598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e745d-cd38-4791-b1d7-28f12321cfe4 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:32.373937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8e655-c7c1-438a-b6cc-7e04dd99101f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:32.377844 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:32.384115 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a11859-2c65-4cf0-9567-c78ca0ae64af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:32.385268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a11859-2c65-4cf0-9567-c78ca0ae64af None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:32.385659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a11859-2c65-4cf0-9567-c78ca0ae64af None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:32.386041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a11859-2c65-4cf0-9567-c78ca0ae64af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:32.387002 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a11859-2c65-4cf0-9567-c78ca0ae64af None admin] RBAC: Adding query filter params (group.id=895fc55542e14dbbaa927216b4e9442f, role.id=22f9190503114c4aba5e898e7a460546, scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:32.387493 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a11859-2c65-4cf0-9567-c78ca0ae64af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:32.390339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a11859-2c65-4cf0-9567-c78ca0ae64af None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:32.399273 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 187/355] 10.222.0.60 () {56 vars in 1376 bytes} [Thu Jul 21 20:38:32 2022] GET /identity/v3/role_assignments?group.id=895fc55542e14dbbaa927216b4e9442f&scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823&role.id=22f9190503114c4aba5e898e7a460546 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:32.399273 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a11859-2c65-4cf0-9567-c78ca0ae64af None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:32.438557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cfb09-3d2f-4d52-82a5-f2a75e06317b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:32.439556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cfb09-3d2f-4d52-82a5-f2a75e06317b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:32.439947 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cfb09-3d2f-4d52-82a5-f2a75e06317b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:32.441245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cfb09-3d2f-4d52-82a5-f2a75e06317b None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:32.441699 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cfb09-3d2f-4d52-82a5-f2a75e06317b None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:32.444262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cfb09-3d2f-4d52-82a5-f2a75e06317b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:32.445861 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 169/356] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:32 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:32.448017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf4430-a1ea-48c4-b807-f40461c896b8 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:32.448017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf4430-a1ea-48c4-b807-f40461c896b8 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:32.448017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf4430-a1ea-48c4-b807-f40461c896b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:32.469112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-b0bf4430-a1ea-48c4-b807-f40461c896b8 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:32.470995 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b0bf4430-a1ea-48c4-b807-f40461c896b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:32.557428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a11859-2c65-4cf0-9567-c78ca0ae64af None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:32.574677 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 170/357] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:32.576580 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf4430-a1ea-48c4-b807-f40461c896b8 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:32.578863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:32.600384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3464a6-491a-4630-b5f5-993668cab525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:32.601606 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3464a6-491a-4630-b5f5-993668cab525 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:32.602107 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3464a6-491a-4630-b5f5-993668cab525 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:32.602286 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3464a6-491a-4630-b5f5-993668cab525 None admin] PATH_INFO: `/v3/users/61fafebcd45c4520bc4c2c8aa4fac9d7` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:32.604506 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3464a6-491a-4630-b5f5-993668cab525 None admin] RBAC: Authorizing `identity:get_user(user_id=61fafebcd45c4520bc4c2c8aa4fac9d7)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:32.606096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3464a6-491a-4630-b5f5-993668cab525 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:32.608147 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 188/358] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:38:32 2022] GET /identity/v3/users/61fafebcd45c4520bc4c2c8aa4fac9d7 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:32.609429 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c97928-8cc6-42e8-8f5c-af5e19e693dd None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:32.609717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c97928-8cc6-42e8-8f5c-af5e19e693dd None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:32.614878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c97928-8cc6-42e8-8f5c-af5e19e693dd None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:32.617940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 189/359] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:32.618800 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c97928-8cc6-42e8-8f5c-af5e19e693dd None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:32.714762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf4430-a1ea-48c4-b807-f40461c896b8 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:32.716491 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:32.758188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeec524-ba34-43ab-b4bd-975bb7c21ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:32.758894 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeec524-ba34-43ab-b4bd-975bb7c21ffb None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:32.759530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeec524-ba34-43ab-b4bd-975bb7c21ffb None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:32.760808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeec524-ba34-43ab-b4bd-975bb7c21ffb None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:32.761604 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeec524-ba34-43ab-b4bd-975bb7c21ffb None admin] RBAC: Adding query filter params (name=member) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:32.761912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeec524-ba34-43ab-b4bd-975bb7c21ffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:32.763908 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeec524-ba34-43ab-b4bd-975bb7c21ffb None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:32.774253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 171/360] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:32.775121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeec524-ba34-43ab-b4bd-975bb7c21ffb None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:32.782972 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c97928-8cc6-42e8-8f5c-af5e19e693dd None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:32.829680 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6a3b2-bd19-4cce-9f4d-fac2018e3ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:32.830652 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6a3b2-bd19-4cce-9f4d-fac2018e3ba3 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:32.830933 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6a3b2-bd19-4cce-9f4d-fac2018e3ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:32.831203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6a3b2-bd19-4cce-9f4d-fac2018e3ba3 None admin] PATH_INFO: `/v3/groups/895fc55542e14dbbaa927216b4e9442f` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:32.832601 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6a3b2-bd19-4cce-9f4d-fac2018e3ba3 None admin] RBAC: Authorizing `identity:get_group(group_id=895fc55542e14dbbaa927216b4e9442f)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:32.834614 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6a3b2-bd19-4cce-9f4d-fac2018e3ba3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:32.836844 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 190/361] 10.222.0.60 () {56 vars in 1180 bytes} [Thu Jul 21 20:38:32 2022] GET /identity/v3/groups/895fc55542e14dbbaa927216b4e9442f => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:32.837558 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6a3b2-bd19-4cce-9f4d-fac2018e3ba3 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:32.920803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeec524-ba34-43ab-b4bd-975bb7c21ffb None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:32.933295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:32.967053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55964a90-ea5f-4bba-90eb-578ab4d615c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:32.968179 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55964a90-ea5f-4bba-90eb-578ab4d615c1 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:32.968751 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55964a90-ea5f-4bba-90eb-578ab4d615c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:32.969032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55964a90-ea5f-4bba-90eb-578ab4d615c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:32.983995 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55964a90-ea5f-4bba-90eb-578ab4d615c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-55964a90-ea5f-4bba-90eb-578ab4d615c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:32.993759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:32.994702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:32.995509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:32.995509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:32.995509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:32.995509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:32.995509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:32.995509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:32.995509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:32.995509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:32.995509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:32.995509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:32.995509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:32.995509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:32.997602 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 172/362] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:38:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:33.002434 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d336d49-53a8-4edd-8e3a-92e212582010 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:33.002820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6a3b2-bd19-4cce-9f4d-fac2018e3ba3 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:33.002931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d336d49-53a8-4edd-8e3a-92e212582010 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:33.003030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d336d49-53a8-4edd-8e3a-92e212582010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:33.024021 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-9d336d49-53a8-4edd-8e3a-92e212582010 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:33.024214 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-9d336d49-53a8-4edd-8e3a-92e212582010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:33.047022 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:33.048062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59645f8b-970a-4b7c-9c99-9894ff345e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:33.048735 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59645f8b-970a-4b7c-9c99-9894ff345e94 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:33.049207 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59645f8b-970a-4b7c-9c99-9894ff345e94 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:33.049306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59645f8b-970a-4b7c-9c99-9894ff345e94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:33.050122 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59645f8b-970a-4b7c-9c99-9894ff345e94 None admin] RBAC: Adding query filter params (role.id=1052f958732d4259a0658b40d1f9c193, user.id=61fafebcd45c4520bc4c2c8aa4fac9d7, scope.system=all) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:33.050438 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59645f8b-970a-4b7c-9c99-9894ff345e94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:33.055621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59645f8b-970a-4b7c-9c99-9894ff345e94 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:33.071100 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 191/363] 10.222.0.60 () {56 vars in 1308 bytes} [Thu Jul 21 20:38:32 2022] GET /identity/v3/role_assignments?user.id=61fafebcd45c4520bc4c2c8aa4fac9d7&scope.system=all&role.id=1052f958732d4259a0658b40d1f9c193 => generated 510 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:33.072699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59645f8b-970a-4b7c-9c99-9894ff345e94 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:33.120404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 173/364] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:33.123465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d336d49-53a8-4edd-8e3a-92e212582010 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:33.230878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59645f8b-970a-4b7c-9c99-9894ff345e94 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:33.246878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:33.275412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d05d8-a054-4d24-9c2f-7e1f3e68ca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:33.276216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d05d8-a054-4d24-9c2f-7e1f3e68ca65 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:33.276661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d05d8-a054-4d24-9c2f-7e1f3e68ca65 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:33.276946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d05d8-a054-4d24-9c2f-7e1f3e68ca65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:33.282141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d336d49-53a8-4edd-8e3a-92e212582010 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:33.329803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a4405-624c-4043-9d7b-a46055250f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:33.330479 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a4405-624c-4043-9d7b-a46055250f2d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:33.330753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a4405-624c-4043-9d7b-a46055250f2d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:33.331025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a4405-624c-4043-9d7b-a46055250f2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:33.331914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a4405-624c-4043-9d7b-a46055250f2d None admin] RBAC: Adding query filter params (group.id=895fc55542e14dbbaa927216b4e9442f, role.id=04597f92ac654fb4ac68dbb783d3dc31, scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:33.332225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a4405-624c-4043-9d7b-a46055250f2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:33.333735 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a4405-624c-4043-9d7b-a46055250f2d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:33.342151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 174/365] 10.222.0.60 () {56 vars in 1376 bytes} [Thu Jul 21 20:38:33 2022] GET /identity/v3/role_assignments?group.id=895fc55542e14dbbaa927216b4e9442f&scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823&role.id=04597f92ac654fb4ac68dbb783d3dc31 => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:33.343013 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a4405-624c-4043-9d7b-a46055250f2d None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:33.375219 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:33.382719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d05d8-a054-4d24-9c2f-7e1f3e68ca65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:33.384704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d05d8-a054-4d24-9c2f-7e1f3e68ca65 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:33.501526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a4405-624c-4043-9d7b-a46055250f2d None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:33.524099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 192/366] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:33.525362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d05d8-a054-4d24-9c2f-7e1f3e68ca65 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:33.547650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:33.552061 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047872c-86df-4670-aeba-4bcd473c5b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:33.552893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047872c-86df-4670-aeba-4bcd473c5b0e None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:33.553385 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047872c-86df-4670-aeba-4bcd473c5b0e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:33.554157 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047872c-86df-4670-aeba-4bcd473c5b0e None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:33.555640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047872c-86df-4670-aeba-4bcd473c5b0e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:33.556066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047872c-86df-4670-aeba-4bcd473c5b0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:33.557586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047872c-86df-4670-aeba-4bcd473c5b0e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:33.567762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 175/367] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:33.568726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047872c-86df-4670-aeba-4bcd473c5b0e None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:33.668790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d05d8-a054-4d24-9c2f-7e1f3e68ca65 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:33.683383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:33.709913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047872c-86df-4670-aeba-4bcd473c5b0e None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:33.718522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab479882-5e10-4a73-9f47-435363c9e04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:33.719320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab479882-5e10-4a73-9f47-435363c9e04c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:33.719720 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab479882-5e10-4a73-9f47-435363c9e04c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:33.720016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab479882-5e10-4a73-9f47-435363c9e04c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:33.740003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab479882-5e10-4a73-9f47-435363c9e04c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-ab479882-5e10-4a73-9f47-435363c9e04c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:33.749106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:33.750013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:33.751213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:33.751213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:33.751213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:33.751213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:33.751213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:33.751213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:33.751213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:33.751213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:33.751213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:33.751213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:33.751213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:38:33.751213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:33.751213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 193/368] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:38:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:33.751213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab479882-5e10-4a73-9f47-435363c9e04c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:33.756158 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a9212-9e05-4deb-b6c5-c28c5282b452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:33.757977 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a9212-9e05-4deb-b6c5-c28c5282b452 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:33.758512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a9212-9e05-4deb-b6c5-c28c5282b452 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:33.758997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a9212-9e05-4deb-b6c5-c28c5282b452 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:33.760460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a9212-9e05-4deb-b6c5-c28c5282b452 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:33.762680 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a9212-9e05-4deb-b6c5-c28c5282b452 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:33.765097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 176/369] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:33 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:33.766627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a9212-9e05-4deb-b6c5-c28c5282b452 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:33.860114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:33.910057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab479882-5e10-4a73-9f47-435363c9e04c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:33.913753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a9212-9e05-4deb-b6c5-c28c5282b452 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:33.956296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f141e-3006-4216-b986-01f8d7114e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:33.957030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b370e-55b2-4258-beb8-bbeb68563a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:33.957726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b370e-55b2-4258-beb8-bbeb68563a07 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:33.958015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b370e-55b2-4258-beb8-bbeb68563a07 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:33.958293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b370e-55b2-4258-beb8-bbeb68563a07 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:33.958776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f141e-3006-4216-b986-01f8d7114e7d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:33.959061 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b370e-55b2-4258-beb8-bbeb68563a07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:33.959371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b370e-55b2-4258-beb8-bbeb68563a07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:33.960201 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f141e-3006-4216-b986-01f8d7114e7d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:33.960695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f141e-3006-4216-b986-01f8d7114e7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:33.961386 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b370e-55b2-4258-beb8-bbeb68563a07 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:33.972894 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 177/370] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:33.973957 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b370e-55b2-4258-beb8-bbeb68563a07 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:33.992792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:34.054918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f141e-3006-4216-b986-01f8d7114e7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:34.056862 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f141e-3006-4216-b986-01f8d7114e7d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:34.121440 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b370e-55b2-4258-beb8-bbeb68563a07 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:34.153696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:34.169832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd2cf0-315e-4db5-a440-0b94e179b939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:34.171039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd2cf0-315e-4db5-a440-0b94e179b939 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:34.171482 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd2cf0-315e-4db5-a440-0b94e179b939 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:34.172054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd2cf0-315e-4db5-a440-0b94e179b939 None admin] PATH_INFO: `/v3/groups/895fc55542e14dbbaa927216b4e9442f` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:34.173644 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd2cf0-315e-4db5-a440-0b94e179b939 None admin] RBAC: Authorizing `identity:get_group(group_id=895fc55542e14dbbaa927216b4e9442f)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:34.175323 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd2cf0-315e-4db5-a440-0b94e179b939 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:34.177410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 178/371] 10.222.0.60 () {56 vars in 1180 bytes} [Thu Jul 21 20:38:33 2022] GET /identity/v3/groups/895fc55542e14dbbaa927216b4e9442f => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:34.178463 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd2cf0-315e-4db5-a440-0b94e179b939 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:34.201200 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 194/372] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:34.202380 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f141e-3006-4216-b986-01f8d7114e7d None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:34.294969 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:34.338745 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd2cf0-315e-4db5-a440-0b94e179b939 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:34.354061 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f141e-3006-4216-b986-01f8d7114e7d None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:34.386359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e875-1f46-461d-8b89-ab45eeaff757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:34.387417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e875-1f46-461d-8b89-ab45eeaff757 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:34.387987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e875-1f46-461d-8b89-ab45eeaff757 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:34.388591 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e875-1f46-461d-8b89-ab45eeaff757 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:34.408850 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407af3d6-2cfb-4fe8-b961-23a922b528b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:34.410676 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407af3d6-2cfb-4fe8-b961-23a922b528b5 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:34.410676 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407af3d6-2cfb-4fe8-b961-23a922b528b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:34.410676 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407af3d6-2cfb-4fe8-b961-23a922b528b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:34.411856 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407af3d6-2cfb-4fe8-b961-23a922b528b5 None admin] RBAC: Adding query filter params (group.id=895fc55542e14dbbaa927216b4e9442f, role.id=22f9190503114c4aba5e898e7a460546, scope.project.id=3fa3d431d9cc468aafe7b895f047ff94) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:34.412297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407af3d6-2cfb-4fe8-b961-23a922b528b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:34.415301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407af3d6-2cfb-4fe8-b961-23a922b528b5 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:34.425931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 195/373] 10.222.0.60 () {56 vars in 1376 bytes} [Thu Jul 21 20:38:34 2022] GET /identity/v3/role_assignments?group.id=895fc55542e14dbbaa927216b4e9442f&scope.project.id=3fa3d431d9cc468aafe7b895f047ff94&role.id=22f9190503114c4aba5e898e7a460546 => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:34.427266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407af3d6-2cfb-4fe8-b961-23a922b528b5 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:34.494118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:34.516390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5e875-1f46-461d-8b89-ab45eeaff757 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:34.519664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5e875-1f46-461d-8b89-ab45eeaff757 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:34.639206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407af3d6-2cfb-4fe8-b961-23a922b528b5 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:34.681514 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:34.695459 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52898c9a-b410-45ac-9847-cc4f58cce608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:34.696445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52898c9a-b410-45ac-9847-cc4f58cce608 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:34.696728 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52898c9a-b410-45ac-9847-cc4f58cce608 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:34.697005 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52898c9a-b410-45ac-9847-cc4f58cce608 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:34.698848 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52898c9a-b410-45ac-9847-cc4f58cce608 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:34.701518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52898c9a-b410-45ac-9847-cc4f58cce608 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:34.707743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 196/374] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:34 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:34.716636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52898c9a-b410-45ac-9847-cc4f58cce608 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:34.720081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 179/375] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:34.728291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e875-1f46-461d-8b89-ab45eeaff757 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:34.902719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52898c9a-b410-45ac-9847-cc4f58cce608 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:34.906181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:34.917730 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e875-1f46-461d-8b89-ab45eeaff757 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:34.948859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5c3cd-1ef1-4c29-abf5-ab9f42b68371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:34.949736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5c3cd-1ef1-4c29-abf5-ab9f42b68371 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:34.950019 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5c3cd-1ef1-4c29-abf5-ab9f42b68371 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:34.950293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5c3cd-1ef1-4c29-abf5-ab9f42b68371 None admin] PATH_INFO: `/v3/groups/895fc55542e14dbbaa927216b4e9442f` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:34.954130 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5c3cd-1ef1-4c29-abf5-ab9f42b68371 None admin] RBAC: Authorizing `identity:get_group(group_id=895fc55542e14dbbaa927216b4e9442f)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:34.955950 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5c3cd-1ef1-4c29-abf5-ab9f42b68371 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:34.958607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 197/376] 10.222.0.60 () {56 vars in 1180 bytes} [Thu Jul 21 20:38:34 2022] GET /identity/v3/groups/895fc55542e14dbbaa927216b4e9442f => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:34.959237 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c632361-6fdd-4c85-8b20-f656a7935caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:34.960170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c632361-6fdd-4c85-8b20-f656a7935caa None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:34.960293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcee399-d31d-4aa2-aff1-2b1dd3b7c536 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:34.960460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c632361-6fdd-4c85-8b20-f656a7935caa None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:34.960648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcee399-d31d-4aa2-aff1-2b1dd3b7c536 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:34.960758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c632361-6fdd-4c85-8b20-f656a7935caa None admin] PATH_INFO: `/v3/projects/a9f8292e70024a6985791def05952106` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:34.960938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcee399-d31d-4aa2-aff1-2b1dd3b7c536 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:34.962033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c632361-6fdd-4c85-8b20-f656a7935caa None admin] RBAC: Authorizing `identity:get_project(project_id=a9f8292e70024a6985791def05952106)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:34.962621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 198/377] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:34.963501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c632361-6fdd-4c85-8b20-f656a7935caa None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:34.966637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcee399-d31d-4aa2-aff1-2b1dd3b7c536 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:34.970871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 180/378] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:34 2022] GET /identity/v3/projects/a9f8292e70024a6985791def05952106 => generated 344 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:34.972278 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0a6d9-b9ac-41fc-9a69-c42abe05cd39 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:34.972565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0a6d9-b9ac-41fc-9a69-c42abe05cd39 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:34.972845 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0a6d9-b9ac-41fc-9a69-c42abe05cd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:34.998101 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-33a0a6d9-b9ac-41fc-9a69-c42abe05cd39 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:34.999631 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-33a0a6d9-b9ac-41fc-9a69-c42abe05cd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:35.052742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:35.086817 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 181/379] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:35.088164 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0a6d9-b9ac-41fc-9a69-c42abe05cd39 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:35.124780 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcee399-d31d-4aa2-aff1-2b1dd3b7c536 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:35.172961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae59889-7b9b-4d8f-b520-67bcf583ce43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:35.174012 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae59889-7b9b-4d8f-b520-67bcf583ce43 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:35.174611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae59889-7b9b-4d8f-b520-67bcf583ce43 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:35.175134 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae59889-7b9b-4d8f-b520-67bcf583ce43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:35.176275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae59889-7b9b-4d8f-b520-67bcf583ce43 None admin] RBAC: Adding query filter params (group.id=895fc55542e14dbbaa927216b4e9442f, role.id=04597f92ac654fb4ac68dbb783d3dc31, scope.project.id=3fa3d431d9cc468aafe7b895f047ff94) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:35.176817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae59889-7b9b-4d8f-b520-67bcf583ce43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:35.178813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae59889-7b9b-4d8f-b520-67bcf583ce43 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:35.186450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 199/380] 10.222.0.60 () {56 vars in 1376 bytes} [Thu Jul 21 20:38:34 2022] GET /identity/v3/role_assignments?group.id=895fc55542e14dbbaa927216b4e9442f&scope.project.id=3fa3d431d9cc468aafe7b895f047ff94&role.id=04597f92ac654fb4ac68dbb783d3dc31 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:35.187828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae59889-7b9b-4d8f-b520-67bcf583ce43 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:35.208489 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0a6d9-b9ac-41fc-9a69-c42abe05cd39 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:35.238363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:35.273520 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ded1f-6f5f-45c3-aac9-de362c6a83dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:35.274435 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ded1f-6f5f-45c3-aac9-de362c6a83dc None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:35.274726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ded1f-6f5f-45c3-aac9-de362c6a83dc None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:35.275013 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ded1f-6f5f-45c3-aac9-de362c6a83dc None admin] PATH_INFO: `/v3/groups/c5825e2dfe954b378367ce6d292c71f8` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:35.297177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ded1f-6f5f-45c3-aac9-de362c6a83dc None admin] RBAC: Authorizing `identity:get_group(group_id=c5825e2dfe954b378367ce6d292c71f8)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:35.298684 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ded1f-6f5f-45c3-aac9-de362c6a83dc None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:35.306150 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 182/381] 10.222.0.60 () {56 vars in 1180 bytes} [Thu Jul 21 20:38:35 2022] GET /identity/v3/groups/c5825e2dfe954b378367ce6d292c71f8 => generated 229 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:35.307128 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ded1f-6f5f-45c3-aac9-de362c6a83dc None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:35.346353 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae59889-7b9b-4d8f-b520-67bcf583ce43 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:35.414530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:35.451607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd96fdb-ce96-46c2-9e88-5f61da0f3c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:35.452370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd96fdb-ce96-46c2-9e88-5f61da0f3c4a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:35.452653 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd96fdb-ce96-46c2-9e88-5f61da0f3c4a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:35.452926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd96fdb-ce96-46c2-9e88-5f61da0f3c4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:35.467418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd96fdb-ce96-46c2-9e88-5f61da0f3c4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-cfd96fdb-ce96-46c2-9e88-5f61da0f3c4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:35.477489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:35.478448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:35.479472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:35.479472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:35.479472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:35.479472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:35.479472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:35.479472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:35.479472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:35.479472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:35.479472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:35.479472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:35.479472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:35.479472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:35.479472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 200/382] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:35.485091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd96fdb-ce96-46c2-9e88-5f61da0f3c4a None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:35.513286 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ded1f-6f5f-45c3-aac9-de362c6a83dc None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:35.553032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8f352-3f06-402a-8b8d-6ab7e182cef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:35.554918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8f352-3f06-402a-8b8d-6ab7e182cef7 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:35.555457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8f352-3f06-402a-8b8d-6ab7e182cef7 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:35.556055 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8f352-3f06-402a-8b8d-6ab7e182cef7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:35.557360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8f352-3f06-402a-8b8d-6ab7e182cef7 None admin] RBAC: Adding query filter params (group.id=c5825e2dfe954b378367ce6d292c71f8, role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=a9f8292e70024a6985791def05952106) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:35.557936 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8f352-3f06-402a-8b8d-6ab7e182cef7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:35.559606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8f352-3f06-402a-8b8d-6ab7e182cef7 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:35.566554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 183/383] 10.222.0.60 () {56 vars in 1376 bytes} [Thu Jul 21 20:38:35 2022] GET /identity/v3/role_assignments?group.id=c5825e2dfe954b378367ce6d292c71f8&scope.project.id=a9f8292e70024a6985791def05952106&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 260 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:35.622117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:35.629029 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd96fdb-ce96-46c2-9e88-5f61da0f3c4a None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:35.690160 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69c323-69dd-4b73-a0c4-c494f5abf80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:35.691095 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69c323-69dd-4b73-a0c4-c494f5abf80e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:35.691524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69c323-69dd-4b73-a0c4-c494f5abf80e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:35.691996 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69c323-69dd-4b73-a0c4-c494f5abf80e None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:35.692905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69c323-69dd-4b73-a0c4-c494f5abf80e None admin] RBAC: Adding query filter params (name=member) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:35.693355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69c323-69dd-4b73-a0c4-c494f5abf80e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:35.695452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69c323-69dd-4b73-a0c4-c494f5abf80e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:35.707977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 201/384] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:35.713340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8f352-3f06-402a-8b8d-6ab7e182cef7 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:35.842016 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8f352-3f06-402a-8b8d-6ab7e182cef7 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:35.845063 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:35.884249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0fdb7-5c67-4efe-96fa-af0723c3b8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:35.885376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0fdb7-5c67-4efe-96fa-af0723c3b8b8 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:35.885830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0fdb7-5c67-4efe-96fa-af0723c3b8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:35.886251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0fdb7-5c67-4efe-96fa-af0723c3b8b8 None admin] PATH_INFO: `/v3/groups/895fc55542e14dbbaa927216b4e9442f` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:35.887669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0fdb7-5c67-4efe-96fa-af0723c3b8b8 None admin] RBAC: Authorizing `identity:get_group(group_id=895fc55542e14dbbaa927216b4e9442f)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:35.889300 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0fdb7-5c67-4efe-96fa-af0723c3b8b8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:35.891460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 184/385] 10.222.0.60 () {56 vars in 1180 bytes} [Thu Jul 21 20:38:35 2022] GET /identity/v3/groups/895fc55542e14dbbaa927216b4e9442f => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:35.900912 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69c323-69dd-4b73-a0c4-c494f5abf80e None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:36.024539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69c323-69dd-4b73-a0c4-c494f5abf80e None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:36.062652 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e855c0b-c800-4c2c-bf21-a2d8e2bd0b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:36.063814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e855c0b-c800-4c2c-bf21-a2d8e2bd0b02 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:36.064245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e855c0b-c800-4c2c-bf21-a2d8e2bd0b02 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:36.064681 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e855c0b-c800-4c2c-bf21-a2d8e2bd0b02 None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:36.066075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e855c0b-c800-4c2c-bf21-a2d8e2bd0b02 None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:36.067760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e855c0b-c800-4c2c-bf21-a2d8e2bd0b02 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:36.070283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 202/386] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:35 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:36.075975 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0fdb7-5c67-4efe-96fa-af0723c3b8b8 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:36.078835 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e05402-e221-4661-816b-f25a61de3b64 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:36.079306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e05402-e221-4661-816b-f25a61de3b64 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:36.079981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e05402-e221-4661-816b-f25a61de3b64 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:36.081520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 203/387] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:36.088117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6abdbd-f0d9-4dcf-879b-3d15d7e21157 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:36.088409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6abdbd-f0d9-4dcf-879b-3d15d7e21157 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:36.088696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6abdbd-f0d9-4dcf-879b-3d15d7e21157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:36.119152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-da6abdbd-f0d9-4dcf-879b-3d15d7e21157 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:36.120415 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-da6abdbd-f0d9-4dcf-879b-3d15d7e21157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:36.163321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:36.259849 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 204/388] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:38:36.277656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6abdbd-f0d9-4dcf-879b-3d15d7e21157 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:36.278773 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0fdb7-5c67-4efe-96fa-af0723c3b8b8 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:36.343325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72142c-05e6-4a43-83a3-16fc8b049909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:36.344436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72142c-05e6-4a43-83a3-16fc8b049909 None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:36.344814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72142c-05e6-4a43-83a3-16fc8b049909 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:36.345160 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72142c-05e6-4a43-83a3-16fc8b049909 None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823/groups/895fc55542e14dbbaa927216b4e9442f/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:36.347425 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72142c-05e6-4a43-83a3-16fc8b049909 None admin] RBAC: Authorizing `identity:create_grant(project_id=32ebd8e81f314e8fa20cb9ba0042a823, group_id=895fc55542e14dbbaa927216b4e9442f, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:36.349000 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72142c-05e6-4a43-83a3-16fc8b049909 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:36.370143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 185/389] 10.222.0.60 () {58 vars in 1442 bytes} [Thu Jul 21 20:38:36 2022] PUT /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823/groups/895fc55542e14dbbaa927216b4e9442f/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:38:36.406263 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6abdbd-f0d9-4dcf-879b-3d15d7e21157 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:36.446719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807137a5-3e4d-4c86-a093-5bdf9fde3fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:36.448000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807137a5-3e4d-4c86-a093-5bdf9fde3fc1 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:36.448472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807137a5-3e4d-4c86-a093-5bdf9fde3fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:36.448931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807137a5-3e4d-4c86-a093-5bdf9fde3fc1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:36.461773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807137a5-3e4d-4c86-a093-5bdf9fde3fc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-807137a5-3e4d-4c86-a093-5bdf9fde3fc1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:36.471014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:36.471991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:36.472758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:36.472758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:36.472758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:36.472758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:36.472758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:36.472758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:36.472758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:36.472758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:36.472758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:36.472758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:36.472758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:36.472758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:36.479907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 205/390] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:38:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:36.483043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72142c-05e6-4a43-83a3-16fc8b049909 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:36.539548 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:36.641601 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72142c-05e6-4a43-83a3-16fc8b049909 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:36.683578 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd760a-884d-4ba4-b642-6f20f9b7c78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:36.684308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd760a-884d-4ba4-b642-6f20f9b7c78c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:36.684595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd760a-884d-4ba4-b642-6f20f9b7c78c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:36.684872 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd760a-884d-4ba4-b642-6f20f9b7c78c None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:36.685644 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd760a-884d-4ba4-b642-6f20f9b7c78c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:36.685954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd760a-884d-4ba4-b642-6f20f9b7c78c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:36.687347 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd760a-884d-4ba4-b642-6f20f9b7c78c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:36.697453 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 186/391] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:36.703412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807137a5-3e4d-4c86-a093-5bdf9fde3fc1 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:36.728616 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:36.864605 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807137a5-3e4d-4c86-a093-5bdf9fde3fc1 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:36.917369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7b675-b5e2-4095-a580-4e7f24e12b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:36.918252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7b675-b5e2-4095-a580-4e7f24e12b33 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:36.918539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7b675-b5e2-4095-a580-4e7f24e12b33 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:36.918832 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7b675-b5e2-4095-a580-4e7f24e12b33 None admin] PATH_INFO: `/v3/groups/895fc55542e14dbbaa927216b4e9442f` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:36.920049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7b675-b5e2-4095-a580-4e7f24e12b33 None admin] RBAC: Authorizing `identity:get_group(group_id=895fc55542e14dbbaa927216b4e9442f)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:36.921518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7b675-b5e2-4095-a580-4e7f24e12b33 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:36.923161 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 206/392] 10.222.0.60 () {56 vars in 1180 bytes} [Thu Jul 21 20:38:36 2022] GET /identity/v3/groups/895fc55542e14dbbaa927216b4e9442f => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:36.928925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd760a-884d-4ba4-b642-6f20f9b7c78c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:37.050990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:37.063641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd760a-884d-4ba4-b642-6f20f9b7c78c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:37.114082 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53c79e-9737-4ea4-bccb-3da59601687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:37.115924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53c79e-9737-4ea4-bccb-3da59601687f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:37.116418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53c79e-9737-4ea4-bccb-3da59601687f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:37.116974 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53c79e-9737-4ea4-bccb-3da59601687f None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:37.118581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53c79e-9737-4ea4-bccb-3da59601687f None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:37.119788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08a7eb-a4f9-4f44-9779-646ac060f3e2 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:37.120445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53c79e-9737-4ea4-bccb-3da59601687f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:37.120857 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08a7eb-a4f9-4f44-9779-646ac060f3e2 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:37.121363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08a7eb-a4f9-4f44-9779-646ac060f3e2 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:37.122770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 187/393] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:36 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:37.123072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 207/394] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:37.128882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0c4cc-6fb9-4558-a01b-7a4c571b3b8a None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:37.129189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0c4cc-6fb9-4558-a01b-7a4c571b3b8a None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:37.129405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08a7eb-a4f9-4f44-9779-646ac060f3e2 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:37.129502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0c4cc-6fb9-4558-a01b-7a4c571b3b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:37.147699 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-44a0c4cc-6fb9-4558-a01b-7a4c571b3b8a None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:37.149587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-44a0c4cc-6fb9-4558-a01b-7a4c571b3b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:37.292785 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 188/395] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:37.312941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:37.342975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08a7eb-a4f9-4f44-9779-646ac060f3e2 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:37.370567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0c4cc-6fb9-4558-a01b-7a4c571b3b8a None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:37.433092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e4af6-cfff-40a6-abd6-7784f98fed03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:37.434009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e4af6-cfff-40a6-abd6-7784f98fed03 None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:37.434480 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e4af6-cfff-40a6-abd6-7784f98fed03 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:37.434586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e4af6-cfff-40a6-abd6-7784f98fed03 None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823/groups/895fc55542e14dbbaa927216b4e9442f/roles/04597f92ac654fb4ac68dbb783d3dc31` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:37.436863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e4af6-cfff-40a6-abd6-7784f98fed03 None admin] RBAC: Authorizing `identity:create_grant(project_id=32ebd8e81f314e8fa20cb9ba0042a823, group_id=895fc55542e14dbbaa927216b4e9442f, role_id=04597f92ac654fb4ac68dbb783d3dc31)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:37.438316 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e4af6-cfff-40a6-abd6-7784f98fed03 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:37.491878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 208/396] 10.222.0.60 () {58 vars in 1442 bytes} [Thu Jul 21 20:38:37 2022] PUT /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823/groups/895fc55542e14dbbaa927216b4e9442f/roles/04597f92ac654fb4ac68dbb783d3dc31 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:38:37.499784 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0c4cc-6fb9-4558-a01b-7a4c571b3b8a None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:37.504035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:37.541605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708bfcd-a73f-4f7a-b222-c847f6eb510f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:37.542780 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708bfcd-a73f-4f7a-b222-c847f6eb510f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:37.543290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708bfcd-a73f-4f7a-b222-c847f6eb510f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:37.543809 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708bfcd-a73f-4f7a-b222-c847f6eb510f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:37.556833 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708bfcd-a73f-4f7a-b222-c847f6eb510f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-6708bfcd-a73f-4f7a-b222-c847f6eb510f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:37.565400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:37.566331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:37.567188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:37.567188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:37.567188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:37.567188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:37.567188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:37.567188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:37.567188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:37.567188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:37.567188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:37.567188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:37.567188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:37.567188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:37.575256 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 189/397] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:38:37.581591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e4af6-cfff-40a6-abd6-7784f98fed03 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:37.801259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e4af6-cfff-40a6-abd6-7784f98fed03 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:37.874256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa0893-5cbb-4056-ab30-20a98eddf347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:37.875520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa0893-5cbb-4056-ab30-20a98eddf347 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:37.876088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa0893-5cbb-4056-ab30-20a98eddf347 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:37.876601 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa0893-5cbb-4056-ab30-20a98eddf347 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:37.878075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa0893-5cbb-4056-ab30-20a98eddf347 None admin] RBAC: Adding query filter params (name=member) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:37.878633 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa0893-5cbb-4056-ab30-20a98eddf347 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:37.881480 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa0893-5cbb-4056-ab30-20a98eddf347 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:37.903024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 209/398] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:37.913079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708bfcd-a73f-4f7a-b222-c847f6eb510f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:38.063470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7bcd32-8d22-4fcd-873e-2d13dfca8e9e None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:38.063470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7bcd32-8d22-4fcd-873e-2d13dfca8e9e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:38.063470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7bcd32-8d22-4fcd-873e-2d13dfca8e9e None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:38.065002 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 210/399] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:38:38.076271 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d4c29-3238-47ad-a274-c3eceb3e6d50 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:38.076761 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d4c29-3238-47ad-a274-c3eceb3e6d50 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:38.077256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d4c29-3238-47ad-a274-c3eceb3e6d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:38.105131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708bfcd-a73f-4f7a-b222-c847f6eb510f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:38.112895 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:38.128835 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e83d4c29-3238-47ad-a274-c3eceb3e6d50 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:38.138148 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e83d4c29-3238-47ad-a274-c3eceb3e6d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:38.148682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54a209-53eb-4f99-91c6-20e71f43d7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:38.150217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54a209-53eb-4f99-91c6-20e71f43d7ce None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:38.150718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54a209-53eb-4f99-91c6-20e71f43d7ce None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:38.151213 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54a209-53eb-4f99-91c6-20e71f43d7ce None admin] PATH_INFO: `/v3/groups/895fc55542e14dbbaa927216b4e9442f` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:38.152731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54a209-53eb-4f99-91c6-20e71f43d7ce None admin] RBAC: Authorizing `identity:get_group(group_id=895fc55542e14dbbaa927216b4e9442f)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:38.154389 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54a209-53eb-4f99-91c6-20e71f43d7ce None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:38.156667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 190/400] 10.222.0.60 () {56 vars in 1180 bytes} [Thu Jul 21 20:38:37 2022] GET /identity/v3/groups/895fc55542e14dbbaa927216b4e9442f => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:38.162327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54a209-53eb-4f99-91c6-20e71f43d7ce None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:38.196309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:38.310190 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 211/401] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:38.332430 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d4c29-3238-47ad-a274-c3eceb3e6d50 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:38.344467 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54a209-53eb-4f99-91c6-20e71f43d7ce None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:38.450644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d4c29-3238-47ad-a274-c3eceb3e6d50 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:38.464836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255abee-2f0c-4ab1-8e08-3cefa1e19983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:38.466341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255abee-2f0c-4ab1-8e08-3cefa1e19983 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:38.466837 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255abee-2f0c-4ab1-8e08-3cefa1e19983 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:38.467334 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255abee-2f0c-4ab1-8e08-3cefa1e19983 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:38.472914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255abee-2f0c-4ab1-8e08-3cefa1e19983 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:38.475704 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255abee-2f0c-4ab1-8e08-3cefa1e19983 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:38.479562 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 191/402] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:38 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:38.485374 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255abee-2f0c-4ab1-8e08-3cefa1e19983 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:38.487518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80f70e-5ce5-4952-af80-6e45199c6f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:38.488479 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80f70e-5ce5-4952-af80-6e45199c6f2a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:38.488769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80f70e-5ce5-4952-af80-6e45199c6f2a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:38.489023 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80f70e-5ce5-4952-af80-6e45199c6f2a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:38.492770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:38.502894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80f70e-5ce5-4952-af80-6e45199c6f2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-6d80f70e-5ce5-4952-af80-6e45199c6f2a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:38.511351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:38.513315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:38.515256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:38.515256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:38.515256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:38.515256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:38.515256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:38.515256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:38.515256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:38.515256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:38.515256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:38.515256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:38.515256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:38.515256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:38.515256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 212/403] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:38:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:38:38.518693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80f70e-5ce5-4952-af80-6e45199c6f2a None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:38.642362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80f70e-5ce5-4952-af80-6e45199c6f2a None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:38.712836 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36b06d-6002-4c07-a1f4-86ff0d150cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:38.713522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36b06d-6002-4c07-a1f4-86ff0d150cc3 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:38.713803 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36b06d-6002-4c07-a1f4-86ff0d150cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:38.714079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36b06d-6002-4c07-a1f4-86ff0d150cc3 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:38.714842 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36b06d-6002-4c07-a1f4-86ff0d150cc3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:38.715144 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36b06d-6002-4c07-a1f4-86ff0d150cc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:38.716563 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36b06d-6002-4c07-a1f4-86ff0d150cc3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:38.723217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:38.734675 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 213/404] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:38.741922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36b06d-6002-4c07-a1f4-86ff0d150cc3 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:38.759350 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255abee-2f0c-4ab1-8e08-3cefa1e19983 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:38.826109 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc168b-a0e1-476f-ac3e-cb716a1af2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:38.827023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc168b-a0e1-476f-ac3e-cb716a1af2bb None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:38.827303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc168b-a0e1-476f-ac3e-cb716a1af2bb None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:38.827575 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc168b-a0e1-476f-ac3e-cb716a1af2bb None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94/groups/895fc55542e14dbbaa927216b4e9442f/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:38.831606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc168b-a0e1-476f-ac3e-cb716a1af2bb None admin] RBAC: Authorizing `identity:create_grant(project_id=3fa3d431d9cc468aafe7b895f047ff94, group_id=895fc55542e14dbbaa927216b4e9442f, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:38.833099 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc168b-a0e1-476f-ac3e-cb716a1af2bb None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:38.859804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 192/405] 10.222.0.60 () {58 vars in 1442 bytes} [Thu Jul 21 20:38:38 2022] PUT /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94/groups/895fc55542e14dbbaa927216b4e9442f/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:38:38.880957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:38.891104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dade69f-3dd5-4462-a98a-526a96069df3 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:38.891423 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dade69f-3dd5-4462-a98a-526a96069df3 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:38.891660 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dade69f-3dd5-4462-a98a-526a96069df3 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:38.893387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 193/406] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:38.901469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098d9f3-089f-4b37-b700-35c390a33755 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:38.902109 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098d9f3-089f-4b37-b700-35c390a33755 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:38.902465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098d9f3-089f-4b37-b700-35c390a33755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:38.923748 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c098d9f3-089f-4b37-b700-35c390a33755 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:38.925396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c098d9f3-089f-4b37-b700-35c390a33755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:38.943134 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36b06d-6002-4c07-a1f4-86ff0d150cc3 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:39.033289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35271f4c-9945-4e31-852d-f5aaad06b6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:39.034547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35271f4c-9945-4e31-852d-f5aaad06b6a3 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:39.035206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35271f4c-9945-4e31-852d-f5aaad06b6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:39.036183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35271f4c-9945-4e31-852d-f5aaad06b6a3 None admin] PATH_INFO: `/v3/groups/895fc55542e14dbbaa927216b4e9442f` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:39.038961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35271f4c-9945-4e31-852d-f5aaad06b6a3 None admin] RBAC: Authorizing `identity:get_group(group_id=895fc55542e14dbbaa927216b4e9442f)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:39.042442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35271f4c-9945-4e31-852d-f5aaad06b6a3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:39.042442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 214/407] 10.222.0.60 () {56 vars in 1180 bytes} [Thu Jul 21 20:38:38 2022] GET /identity/v3/groups/895fc55542e14dbbaa927216b4e9442f => generated 226 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:39.050785 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35271f4c-9945-4e31-852d-f5aaad06b6a3 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:39.077754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 194/408] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:38:39.107837 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098d9f3-089f-4b37-b700-35c390a33755 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:39.185498 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35271f4c-9945-4e31-852d-f5aaad06b6a3 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:39.212683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:39.227827 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:39.240062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b921a85-ff83-42c2-a2e1-86ba3f6b3879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:39.241700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098d9f3-089f-4b37-b700-35c390a33755 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:39.242065 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b921a85-ff83-42c2-a2e1-86ba3f6b3879 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:39.242544 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b921a85-ff83-42c2-a2e1-86ba3f6b3879 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:39.243014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b921a85-ff83-42c2-a2e1-86ba3f6b3879 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:39.245695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b921a85-ff83-42c2-a2e1-86ba3f6b3879 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:39.247440 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b921a85-ff83-42c2-a2e1-86ba3f6b3879 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:39.249839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 215/409] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:39 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:39.252006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a60444-4a27-4cdb-b865-018a6957ab64 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:39.252516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a60444-4a27-4cdb-b865-018a6957ab64 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:39.252991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a60444-4a27-4cdb-b865-018a6957ab64 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:39.254859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 216/410] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:39.260673 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a60444-4a27-4cdb-b865-018a6957ab64 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:39.300002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed725d-1daa-4b31-a37c-ceaf897db63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:39.302446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed725d-1daa-4b31-a37c-ceaf897db63c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:39.303945 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed725d-1daa-4b31-a37c-ceaf897db63c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:39.304222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed725d-1daa-4b31-a37c-ceaf897db63c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:39.320262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed725d-1daa-4b31-a37c-ceaf897db63c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-f3ed725d-1daa-4b31-a37c-ceaf897db63c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:39.332311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:39.333263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:39.334023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:39.334023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:39.334023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:39.334023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:39.334023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:39.334023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:39.334023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:39.334023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:39.334023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:39.334023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:39.334023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:38:39.334023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:39.339555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 195/411] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:38:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:39.341650 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76286b32-9ee4-4a8b-8a54-f8109ec317fe None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:39.341939 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76286b32-9ee4-4a8b-8a54-f8109ec317fe None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:39.342218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76286b32-9ee4-4a8b-8a54-f8109ec317fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:39.370418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-76286b32-9ee4-4a8b-8a54-f8109ec317fe None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:39.371633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-76286b32-9ee4-4a8b-8a54-f8109ec317fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:39.412313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a60444-4a27-4cdb-b865-018a6957ab64 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:39.457508 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a79909-87c5-4a91-8b2d-76271d188ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:39.458732 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a79909-87c5-4a91-8b2d-76271d188ab3 None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:39.459223 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a79909-87c5-4a91-8b2d-76271d188ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:39.460170 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a79909-87c5-4a91-8b2d-76271d188ab3 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94/groups/895fc55542e14dbbaa927216b4e9442f/roles/04597f92ac654fb4ac68dbb783d3dc31` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:39.462023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 196/412] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:39.463064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76286b32-9ee4-4a8b-8a54-f8109ec317fe None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:39.468775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a79909-87c5-4a91-8b2d-76271d188ab3 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fa3d431d9cc468aafe7b895f047ff94, group_id=895fc55542e14dbbaa927216b4e9442f, role_id=04597f92ac654fb4ac68dbb783d3dc31)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:39.470397 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a79909-87c5-4a91-8b2d-76271d188ab3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:39.498664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 217/413] 10.222.0.60 () {58 vars in 1442 bytes} [Thu Jul 21 20:38:39 2022] PUT /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94/groups/895fc55542e14dbbaa927216b4e9442f/roles/04597f92ac654fb4ac68dbb783d3dc31 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:38:39.504665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a79909-87c5-4a91-8b2d-76271d188ab3 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:39.578692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:39.612223 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:39.671782 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76286b32-9ee4-4a8b-8a54-f8109ec317fe None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:39.710635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a79909-87c5-4a91-8b2d-76271d188ab3 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:39.740022 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77554938-bd66-44b0-a30f-5deba9ac7792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:39.741478 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77554938-bd66-44b0-a30f-5deba9ac7792 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:39.741969 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77554938-bd66-44b0-a30f-5deba9ac7792 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:39.742440 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77554938-bd66-44b0-a30f-5deba9ac7792 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:39.743432 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77554938-bd66-44b0-a30f-5deba9ac7792 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:39.744689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77554938-bd66-44b0-a30f-5deba9ac7792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:39.746345 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77554938-bd66-44b0-a30f-5deba9ac7792 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:39.759223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 197/414] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:39.764806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77554938-bd66-44b0-a30f-5deba9ac7792 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:39.766804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53218593-a188-4272-8114-7b6a29a51cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:39.767704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53218593-a188-4272-8114-7b6a29a51cc1 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:39.768034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53218593-a188-4272-8114-7b6a29a51cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:39.768314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53218593-a188-4272-8114-7b6a29a51cc1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:39.786632 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53218593-a188-4272-8114-7b6a29a51cc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-53218593-a188-4272-8114-7b6a29a51cc1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:39.797981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:39.799335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:39.800817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:39.800817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:39.800817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:39.800817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:39.800817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:39.800817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:39.800817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:39.800817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:39.800817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:39.800817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:39.800817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:39.800817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:39.800817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 218/415] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:39.807828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53218593-a188-4272-8114-7b6a29a51cc1 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:39.904435 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77554938-bd66-44b0-a30f-5deba9ac7792 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:39.935805 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:39.948790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53218593-a188-4272-8114-7b6a29a51cc1 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:39.949001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834065a5-c9f1-4772-be7d-91299c88ec96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:39.949916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834065a5-c9f1-4772-be7d-91299c88ec96 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:39.950202 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834065a5-c9f1-4772-be7d-91299c88ec96 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:39.950482 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834065a5-c9f1-4772-be7d-91299c88ec96 None admin] PATH_INFO: `/v3/groups/c5825e2dfe954b378367ce6d292c71f8` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:39.953063 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834065a5-c9f1-4772-be7d-91299c88ec96 None admin] RBAC: Authorizing `identity:get_group(group_id=c5825e2dfe954b378367ce6d292c71f8)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:39.953999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834065a5-c9f1-4772-be7d-91299c88ec96 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:39.960564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 198/416] 10.222.0.60 () {56 vars in 1180 bytes} [Thu Jul 21 20:38:39 2022] GET /identity/v3/groups/c5825e2dfe954b378367ce6d292c71f8 => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:39.964477 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834065a5-c9f1-4772-be7d-91299c88ec96 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:39.966571 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:39.989058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77148ba6-dc00-4459-8e86-c5d2f3f89efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:39.990132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77148ba6-dc00-4459-8e86-c5d2f3f89efc None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:39.991157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77148ba6-dc00-4459-8e86-c5d2f3f89efc None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:39.991653 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77148ba6-dc00-4459-8e86-c5d2f3f89efc None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:39.992647 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77148ba6-dc00-4459-8e86-c5d2f3f89efc None admin] RBAC: Adding query filter params (name=member) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:39.993199 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77148ba6-dc00-4459-8e86-c5d2f3f89efc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:39.994858 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77148ba6-dc00-4459-8e86-c5d2f3f89efc None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:40.004720 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 219/417] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:40.016783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77148ba6-dc00-4459-8e86-c5d2f3f89efc None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:40.098344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834065a5-c9f1-4772-be7d-91299c88ec96 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:40.132118 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d8b95-d4bc-488c-a883-8fa3d382221b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:40.133218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d8b95-d4bc-488c-a883-8fa3d382221b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:40.133502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d8b95-d4bc-488c-a883-8fa3d382221b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:40.133779 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d8b95-d4bc-488c-a883-8fa3d382221b None admin] PATH_INFO: `/v3/projects/a9f8292e70024a6985791def05952106` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:40.135064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d8b95-d4bc-488c-a883-8fa3d382221b None admin] RBAC: Authorizing `identity:get_project(project_id=a9f8292e70024a6985791def05952106)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:40.136796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d8b95-d4bc-488c-a883-8fa3d382221b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:40.139191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 199/418] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:39 2022] GET /identity/v3/projects/a9f8292e70024a6985791def05952106 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:40.144411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d8b95-d4bc-488c-a883-8fa3d382221b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:40.229493 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:40.273886 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d8b95-d4bc-488c-a883-8fa3d382221b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:40.313223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19b2ff-b19e-44f8-a1c5-60f0f76e1fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:40.314113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19b2ff-b19e-44f8-a1c5-60f0f76e1fa8 None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:40.314396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19b2ff-b19e-44f8-a1c5-60f0f76e1fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:40.314674 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19b2ff-b19e-44f8-a1c5-60f0f76e1fa8 None admin] PATH_INFO: `/v3/projects/a9f8292e70024a6985791def05952106/groups/c5825e2dfe954b378367ce6d292c71f8/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:40.315192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77148ba6-dc00-4459-8e86-c5d2f3f89efc None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:40.319181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19b2ff-b19e-44f8-a1c5-60f0f76e1fa8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9f8292e70024a6985791def05952106, group_id=c5825e2dfe954b378367ce6d292c71f8, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:40.323408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19b2ff-b19e-44f8-a1c5-60f0f76e1fa8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:40.349679 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 200/419] 10.222.0.60 () {58 vars in 1442 bytes} [Thu Jul 21 20:38:40 2022] PUT /identity/v3/projects/a9f8292e70024a6985791def05952106/groups/c5825e2dfe954b378367ce6d292c71f8/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:38:40.365606 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b010cf7-0cc4-45be-bd08-f149ab3769f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:40.366637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b010cf7-0cc4-45be-bd08-f149ab3769f4 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:40.367148 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b010cf7-0cc4-45be-bd08-f149ab3769f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:40.367633 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b010cf7-0cc4-45be-bd08-f149ab3769f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:40.427206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:40.433589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8616b27-b5bd-4277-8fb3-8f0a033841bd None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:40.433939 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8616b27-b5bd-4277-8fb3-8f0a033841bd None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:40.434384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8616b27-b5bd-4277-8fb3-8f0a033841bd None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:40.444831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 201/420] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:40 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:40.450919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035c910-6e6c-41c3-97b1-716ac021eca3 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:40.451235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035c910-6e6c-41c3-97b1-716ac021eca3 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:40.451485 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035c910-6e6c-41c3-97b1-716ac021eca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:40.458948 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b010cf7-0cc4-45be-bd08-f149ab3769f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:40.461700 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b010cf7-0cc4-45be-bd08-f149ab3769f4 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:40.495600 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-1035c910-6e6c-41c3-97b1-716ac021eca3 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:40.496903 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1035c910-6e6c-41c3-97b1-716ac021eca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:40.591216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 220/421] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:40.598170 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b010cf7-0cc4-45be-bd08-f149ab3769f4 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:40.614704 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 202/422] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:40.639512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035c910-6e6c-41c3-97b1-716ac021eca3 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:40.688490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:40.716477 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b010cf7-0cc4-45be-bd08-f149ab3769f4 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:40.756559 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec39dc-deec-4796-b376-2b018cd355f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:40.757650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec39dc-deec-4796-b376-2b018cd355f2 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:40.757942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec39dc-deec-4796-b376-2b018cd355f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:40.758226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec39dc-deec-4796-b376-2b018cd355f2 None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:40.759681 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec39dc-deec-4796-b376-2b018cd355f2 None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:40.761634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec39dc-deec-4796-b376-2b018cd355f2 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:40.763915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 221/423] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:40 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:40.773475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec39dc-deec-4796-b376-2b018cd355f2 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:40.822425 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:40.877508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035c910-6e6c-41c3-97b1-716ac021eca3 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:40.915630 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5e35f-1ebb-4988-bb6d-177cc8e46a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:40.916759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5e35f-1ebb-4988-bb6d-177cc8e46a78 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:40.917193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5e35f-1ebb-4988-bb6d-177cc8e46a78 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:40.917605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5e35f-1ebb-4988-bb6d-177cc8e46a78 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:40.951415 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f5e35f-1ebb-4988-bb6d-177cc8e46a78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:40.960995 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec39dc-deec-4796-b376-2b018cd355f2 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-12f5e35f-1ebb-4988-bb6d-177cc8e46a78 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:40.966927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:40.967847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:40.968618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:40.968618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:40.968618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:40.968618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:40.968618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:40.968618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:40.968618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:40.968618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:40.968618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:40.968618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:40.968618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:40.968618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:40.968618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 203/424] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:38:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:40.981948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5e35f-1ebb-4988-bb6d-177cc8e46a78 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:41.050089 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b9c49-8750-43c0-ba55-188ed3809a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:41.050877 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b9c49-8750-43c0-ba55-188ed3809a1c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:41.051159 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b9c49-8750-43c0-ba55-188ed3809a1c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:41.051433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b9c49-8750-43c0-ba55-188ed3809a1c None admin] PATH_INFO: `/v3/groups/895fc55542e14dbbaa927216b4e9442f` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:41.056759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b9c49-8750-43c0-ba55-188ed3809a1c None admin] RBAC: Authorizing `identity:get_group(group_id=895fc55542e14dbbaa927216b4e9442f)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:41.058219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b9c49-8750-43c0-ba55-188ed3809a1c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:41.066942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 222/425] 10.222.0.60 () {56 vars in 1180 bytes} [Thu Jul 21 20:38:40 2022] GET /identity/v3/groups/895fc55542e14dbbaa927216b4e9442f => generated 226 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:41.073905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b9c49-8750-43c0-ba55-188ed3809a1c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:41.096468 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:41.229687 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5e35f-1ebb-4988-bb6d-177cc8e46a78 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:41.245509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b9c49-8750-43c0-ba55-188ed3809a1c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:41.266242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:41.303770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cb18a-f712-401c-8c65-6f79b9d786ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:41.304999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cb18a-f712-401c-8c65-6f79b9d786ff None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:41.305285 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cb18a-f712-401c-8c65-6f79b9d786ff None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:41.305561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cb18a-f712-401c-8c65-6f79b9d786ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:41.306417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4cb18a-f712-401c-8c65-6f79b9d786ff None admin] RBAC: Adding query filter params (group.id=895fc55542e14dbbaa927216b4e9442f, role.id=22f9190503114c4aba5e898e7a460546, scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:41.306731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4cb18a-f712-401c-8c65-6f79b9d786ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:41.309155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4cb18a-f712-401c-8c65-6f79b9d786ff None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:41.313795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5d8ae-26ed-4b2a-a9cc-27916ddcc68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:41.314479 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5d8ae-26ed-4b2a-a9cc-27916ddcc68b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:41.314758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5d8ae-26ed-4b2a-a9cc-27916ddcc68b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:41.315032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5d8ae-26ed-4b2a-a9cc-27916ddcc68b None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:41.315841 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5d8ae-26ed-4b2a-a9cc-27916ddcc68b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:41.316148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5d8ae-26ed-4b2a-a9cc-27916ddcc68b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:41.317563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5d8ae-26ed-4b2a-a9cc-27916ddcc68b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:41.326500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 223/426] 10.222.0.60 () {56 vars in 1376 bytes} [Thu Jul 21 20:38:41 2022] GET /identity/v3/role_assignments?group.id=895fc55542e14dbbaa927216b4e9442f&scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823&role.id=22f9190503114c4aba5e898e7a460546 => generated 611 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:41.339938 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 204/427] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:41.346093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cb18a-f712-401c-8c65-6f79b9d786ff None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:41.416708 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707821f6-dd73-4c38-b18b-6d3d07adb260 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:41.416708 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707821f6-dd73-4c38-b18b-6d3d07adb260 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:41.416708 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707821f6-dd73-4c38-b18b-6d3d07adb260 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:41.416708 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 205/428] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:38:41.425261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421064c7-d52c-43bf-b957-b26ffa934cf3 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:41.425565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421064c7-d52c-43bf-b957-b26ffa934cf3 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:41.425813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421064c7-d52c-43bf-b957-b26ffa934cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:41.455325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-421064c7-d52c-43bf-b957-b26ffa934cf3 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:41.456606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-421064c7-d52c-43bf-b957-b26ffa934cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:41.508144 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:41.574933 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cb18a-f712-401c-8c65-6f79b9d786ff None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:41.592187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 206/429] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:38:41.619122 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6482e-eaf6-441a-8452-0c808744d8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:41.620157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6482e-eaf6-441a-8452-0c808744d8c3 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:41.620448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6482e-eaf6-441a-8452-0c808744d8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:41.620787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6482e-eaf6-441a-8452-0c808744d8c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:41.621159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421064c7-d52c-43bf-b957-b26ffa934cf3 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:41.713151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6482e-eaf6-441a-8452-0c808744d8c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:41.715271 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6482e-eaf6-441a-8452-0c808744d8c3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:41.759821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421064c7-d52c-43bf-b957-b26ffa934cf3 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:41.772992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:41.807737 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d6d0e-7216-46d2-bcee-dc35ac038fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:41.808897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d6d0e-7216-46d2-bcee-dc35ac038fa5 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:41.809400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d6d0e-7216-46d2-bcee-dc35ac038fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:41.810123 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d6d0e-7216-46d2-bcee-dc35ac038fa5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:41.828078 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:41.833274 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d6d0e-7216-46d2-bcee-dc35ac038fa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-389d6d0e-7216-46d2-bcee-dc35ac038fa5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:41.844525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:41.845470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:41.846245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:41.846245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:41.846245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:41.846245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:41.846245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:41.846245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:41.846245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:41.846245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:41.846245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:41.846245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:41.846245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 20:38:41.846245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:41.849299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 207/430] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:41.857290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d6d0e-7216-46d2-bcee-dc35ac038fa5 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:41.873181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 224/431] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:41.876652 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6482e-eaf6-441a-8452-0c808744d8c3 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:41.988933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d6d0e-7216-46d2-bcee-dc35ac038fa5 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:42.016471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6482e-eaf6-441a-8452-0c808744d8c3 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:42.032588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f01eb-717d-4553-a359-fc63c309fff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:42.033584 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f01eb-717d-4553-a359-fc63c309fff4 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:42.033981 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f01eb-717d-4553-a359-fc63c309fff4 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:42.034377 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f01eb-717d-4553-a359-fc63c309fff4 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:42.035346 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f01eb-717d-4553-a359-fc63c309fff4 None admin] RBAC: Adding query filter params (name=member) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:42.035782 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f01eb-717d-4553-a359-fc63c309fff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:42.037863 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f01eb-717d-4553-a359-fc63c309fff4 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:42.048985 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 208/432] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:38:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:42.054717 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f01eb-717d-4553-a359-fc63c309fff4 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:42.059620 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5960b82-5fbc-4f84-9c65-41ec83422d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:42.061807 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5960b82-5fbc-4f84-9c65-41ec83422d97 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:42.062247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5960b82-5fbc-4f84-9c65-41ec83422d97 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:42.062930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5960b82-5fbc-4f84-9c65-41ec83422d97 None admin] PATH_INFO: `/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:42.064878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5960b82-5fbc-4f84-9c65-41ec83422d97 None admin] RBAC: Authorizing `identity:get_project(project_id=32ebd8e81f314e8fa20cb9ba0042a823)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:42.066722 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5960b82-5fbc-4f84-9c65-41ec83422d97 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:42.074472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 225/433] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:41 2022] GET /identity/v3/projects/32ebd8e81f314e8fa20cb9ba0042a823 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:42.077058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5960b82-5fbc-4f84-9c65-41ec83422d97 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:42.118937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:42.148004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:42.189386 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f01eb-717d-4553-a359-fc63c309fff4 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:42.212911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5960b82-5fbc-4f84-9c65-41ec83422d97 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:42.231232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa323d7-57e4-41ec-bc10-b15731929f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:42.231916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa323d7-57e4-41ec-bc10-b15731929f64 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:42.232202 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa323d7-57e4-41ec-bc10-b15731929f64 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:42.232479 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa323d7-57e4-41ec-bc10-b15731929f64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:42.262244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668c675-5af7-4767-939e-748a19cea33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:42.263082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668c675-5af7-4767-939e-748a19cea33e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:42.263356 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668c675-5af7-4767-939e-748a19cea33e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:42.265496 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668c675-5af7-4767-939e-748a19cea33e None admin] PATH_INFO: `/v3/groups/895fc55542e14dbbaa927216b4e9442f` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:42.266492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668c675-5af7-4767-939e-748a19cea33e None admin] RBAC: Authorizing `identity:get_group(group_id=895fc55542e14dbbaa927216b4e9442f)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:42.267837 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668c675-5af7-4767-939e-748a19cea33e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:42.269864 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 226/434] 10.222.0.60 () {56 vars in 1180 bytes} [Thu Jul 21 20:38:42 2022] GET /identity/v3/groups/895fc55542e14dbbaa927216b4e9442f => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:42.271067 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d3207-a343-4c46-beca-ee26f967a8ce None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:42.271364 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d3207-a343-4c46-beca-ee26f967a8ce None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:42.271797 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d3207-a343-4c46-beca-ee26f967a8ce None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:42.277816 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 227/435] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:42.278835 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d3207-a343-4c46-beca-ee26f967a8ce None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:42.333634 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa323d7-57e4-41ec-bc10-b15731929f64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:42.336027 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa323d7-57e4-41ec-bc10-b15731929f64 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:42.372201 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:42.396006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d3207-a343-4c46-beca-ee26f967a8ce None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:42.414593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:42.442233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f81db-3946-453e-8206-65c88f6644ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:42.442956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f81db-3946-453e-8206-65c88f6644ee None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:42.443238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f81db-3946-453e-8206-65c88f6644ee None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:42.443511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f81db-3946-453e-8206-65c88f6644ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:42.445300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f81db-3946-453e-8206-65c88f6644ee None admin] RBAC: Adding query filter params (group.id=895fc55542e14dbbaa927216b4e9442f, role.id=04597f92ac654fb4ac68dbb783d3dc31, scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:42.445615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f81db-3946-453e-8206-65c88f6644ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:42.447075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f81db-3946-453e-8206-65c88f6644ee None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:42.463757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 228/436] 10.222.0.60 () {56 vars in 1376 bytes} [Thu Jul 21 20:38:42 2022] GET /identity/v3/role_assignments?group.id=895fc55542e14dbbaa927216b4e9442f&scope.project.id=32ebd8e81f314e8fa20cb9ba0042a823&role.id=04597f92ac654fb4ac68dbb783d3dc31 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:42.463757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb84db-e564-4098-950e-3b531efe7bb3 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:42.463757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb84db-e564-4098-950e-3b531efe7bb3 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:42.463757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb84db-e564-4098-950e-3b531efe7bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:42.475341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 209/437] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:42.481791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa323d7-57e4-41ec-bc10-b15731929f64 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:42.481963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-31fb84db-e564-4098-950e-3b531efe7bb3 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:42.483066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-31fb84db-e564-4098-950e-3b531efe7bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:42.595799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 229/438] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:42.613505 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa323d7-57e4-41ec-bc10-b15731929f64 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:42.619857 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb84db-e564-4098-950e-3b531efe7bb3 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:42.659267 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:42.671155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3797a27-c180-41a1-a9a8-90c6f42fe899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:42.672373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3797a27-c180-41a1-a9a8-90c6f42fe899 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:42.672661 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3797a27-c180-41a1-a9a8-90c6f42fe899 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:42.672940 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3797a27-c180-41a1-a9a8-90c6f42fe899 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:42.675459 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3797a27-c180-41a1-a9a8-90c6f42fe899 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:42.676977 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3797a27-c180-41a1-a9a8-90c6f42fe899 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:42.679515 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 210/439] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:42 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:42.685135 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3797a27-c180-41a1-a9a8-90c6f42fe899 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:42.770280 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb84db-e564-4098-950e-3b531efe7bb3 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:42.781787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:42.822714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016ef6b-80d4-444c-b381-b0df916d0440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:42.824938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016ef6b-80d4-444c-b381-b0df916d0440 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:42.824938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016ef6b-80d4-444c-b381-b0df916d0440 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:42.824938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016ef6b-80d4-444c-b381-b0df916d0440 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:42.833021 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3797a27-c180-41a1-a9a8-90c6f42fe899 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:42.837921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016ef6b-80d4-444c-b381-b0df916d0440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-5016ef6b-80d4-444c-b381-b0df916d0440 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:42.845507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:42.846432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:42.847215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:42.847215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:42.847215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:42.847215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:42.847215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:42.847215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:42.847215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:42.847215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:42.847215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:42.847215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:42.847215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:42.847215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:42.847215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 230/440] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:38:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:38:42.848911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da376a0-3a35-4797-873e-b0ea1c3c6449 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:42.848911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da376a0-3a35-4797-873e-b0ea1c3c6449 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:42.849178 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da376a0-3a35-4797-873e-b0ea1c3c6449 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:42.851593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 231/441] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:42.857584 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815168a-4ebe-48d1-9fdc-60f9017904f4 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:42.857875 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815168a-4ebe-48d1-9fdc-60f9017904f4 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:42.858159 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815168a-4ebe-48d1-9fdc-60f9017904f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:42.876526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-1815168a-4ebe-48d1-9fdc-60f9017904f4 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:42.877733 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1815168a-4ebe-48d1-9fdc-60f9017904f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:42.881381 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50103b92-e94e-4433-8260-45e127b7856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:42.882793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50103b92-e94e-4433-8260-45e127b7856e None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:42.883058 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50103b92-e94e-4433-8260-45e127b7856e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:42.883470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50103b92-e94e-4433-8260-45e127b7856e None admin] PATH_INFO: `/v3/groups/895fc55542e14dbbaa927216b4e9442f` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:42.884609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50103b92-e94e-4433-8260-45e127b7856e None admin] RBAC: Authorizing `identity:get_group(group_id=895fc55542e14dbbaa927216b4e9442f)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:42.886133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50103b92-e94e-4433-8260-45e127b7856e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:42.887750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 211/442] 10.222.0.60 () {56 vars in 1180 bytes} [Thu Jul 21 20:38:42 2022] GET /identity/v3/groups/895fc55542e14dbbaa927216b4e9442f => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:42.888570 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50103b92-e94e-4433-8260-45e127b7856e None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:42.957898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 232/443] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:38:42.958747 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815168a-4ebe-48d1-9fdc-60f9017904f4 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:42.967958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:43.006599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50103b92-e94e-4433-8260-45e127b7856e None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:43.048217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba070203-4cc2-400c-b718-41386b02089c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:43.049301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba070203-4cc2-400c-b718-41386b02089c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:43.049584 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba070203-4cc2-400c-b718-41386b02089c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:43.049861 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba070203-4cc2-400c-b718-41386b02089c None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:43.050623 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba070203-4cc2-400c-b718-41386b02089c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:43.050922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba070203-4cc2-400c-b718-41386b02089c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:43.052319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba070203-4cc2-400c-b718-41386b02089c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:43.066522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 212/444] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:43.066522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba070203-4cc2-400c-b718-41386b02089c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:43.073512 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:43.080214 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815168a-4ebe-48d1-9fdc-60f9017904f4 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:43.117174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744e32f-a20f-4ae0-9749-07fc387c278e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:43.117947 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e32f-a20f-4ae0-9749-07fc387c278e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:43.118295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e32f-a20f-4ae0-9749-07fc387c278e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:43.118634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e32f-a20f-4ae0-9749-07fc387c278e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:43.119532 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744e32f-a20f-4ae0-9749-07fc387c278e None admin] RBAC: Adding query filter params (group.id=895fc55542e14dbbaa927216b4e9442f, role.id=22f9190503114c4aba5e898e7a460546, scope.project.id=3fa3d431d9cc468aafe7b895f047ff94) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:43.119934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744e32f-a20f-4ae0-9749-07fc387c278e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:43.121393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744e32f-a20f-4ae0-9749-07fc387c278e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:43.127331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 233/445] 10.222.0.60 () {56 vars in 1376 bytes} [Thu Jul 21 20:38:42 2022] GET /identity/v3/role_assignments?group.id=895fc55542e14dbbaa927216b4e9442f&scope.project.id=3fa3d431d9cc468aafe7b895f047ff94&role.id=22f9190503114c4aba5e898e7a460546 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:43.128172 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744e32f-a20f-4ae0-9749-07fc387c278e None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:43.177869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba070203-4cc2-400c-b718-41386b02089c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:43.208036 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:43.218202 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82732dfc-d996-4654-99bf-bf674906db1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:43.219030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82732dfc-d996-4654-99bf-bf674906db1e None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:43.219314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82732dfc-d996-4654-99bf-bf674906db1e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:43.219589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82732dfc-d996-4654-99bf-bf674906db1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:43.232465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82732dfc-d996-4654-99bf-bf674906db1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-82732dfc-d996-4654-99bf-bf674906db1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:43.240778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:43.241968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:43.242933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:43.242933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:43.242933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:43.242933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:43.242933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:43.242933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:43.242933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:43.242933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:43.242933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:43.242933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:43.242933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:38:43.242933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:43.247230 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 213/446] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:38:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:43.248231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82732dfc-d996-4654-99bf-bf674906db1e None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:43.262272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744e32f-a20f-4ae0-9749-07fc387c278e None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:43.318391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550302d6-dc73-48c1-b6e1-1d5f5256295c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:43.320307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550302d6-dc73-48c1-b6e1-1d5f5256295c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:43.320787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550302d6-dc73-48c1-b6e1-1d5f5256295c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:43.321242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550302d6-dc73-48c1-b6e1-1d5f5256295c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:43.369584 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:43.409229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82732dfc-d996-4654-99bf-bf674906db1e None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:43.431265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550302d6-dc73-48c1-b6e1-1d5f5256295c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:43.432776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550302d6-dc73-48c1-b6e1-1d5f5256295c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:43.464182 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480549b8-6b94-4ecf-8e2f-df6d0b360021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:43.464792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480549b8-6b94-4ecf-8e2f-df6d0b360021 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:43.465079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480549b8-6b94-4ecf-8e2f-df6d0b360021 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:43.465361 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480549b8-6b94-4ecf-8e2f-df6d0b360021 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:43.466119 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480549b8-6b94-4ecf-8e2f-df6d0b360021 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:43.466428 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480549b8-6b94-4ecf-8e2f-df6d0b360021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:43.467966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480549b8-6b94-4ecf-8e2f-df6d0b360021 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:43.478122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 214/447] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:43.484231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480549b8-6b94-4ecf-8e2f-df6d0b360021 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:43.555806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:43.568289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 234/448] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:43.573994 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550302d6-dc73-48c1-b6e1-1d5f5256295c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:43.616988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480549b8-6b94-4ecf-8e2f-df6d0b360021 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:43.638200 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:43.653073 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e2ef9-c87e-4a42-b6f7-6729421b7e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:43.653739 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e2ef9-c87e-4a42-b6f7-6729421b7e54 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:43.654033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e2ef9-c87e-4a42-b6f7-6729421b7e54 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:43.654307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e2ef9-c87e-4a42-b6f7-6729421b7e54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:43.682656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550302d6-dc73-48c1-b6e1-1d5f5256295c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:43.724241 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97621c80-e5bc-412d-9667-0a948b935b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:43.725089 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97621c80-e5bc-412d-9667-0a948b935b13 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:43.725566 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97621c80-e5bc-412d-9667-0a948b935b13 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:43.726062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97621c80-e5bc-412d-9667-0a948b935b13 None admin] PATH_INFO: `/v3/projects/3fa3d431d9cc468aafe7b895f047ff94` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:43.727867 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e2ef9-c87e-4a42-b6f7-6729421b7e54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:43.728022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97621c80-e5bc-412d-9667-0a948b935b13 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa3d431d9cc468aafe7b895f047ff94)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:43.729725 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97621c80-e5bc-412d-9667-0a948b935b13 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:43.729840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e2ef9-c87e-4a42-b6f7-6729421b7e54 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:43.731568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 235/449] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:43 2022] GET /identity/v3/projects/3fa3d431d9cc468aafe7b895f047ff94 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:43.737245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97621c80-e5bc-412d-9667-0a948b935b13 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:43.782982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:43.841193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 215/450] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 20:38:43.841490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97621c80-e5bc-412d-9667-0a948b935b13 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:43.846713 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e2ef9-c87e-4a42-b6f7-6729421b7e54 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:43.873436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:43.882592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c883f1-be83-46d3-b5db-df92ccb09b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:43.883456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c883f1-be83-46d3-b5db-df92ccb09b0b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:43.883755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c883f1-be83-46d3-b5db-df92ccb09b0b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:43.884042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c883f1-be83-46d3-b5db-df92ccb09b0b None admin] PATH_INFO: `/v3/groups/895fc55542e14dbbaa927216b4e9442f` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:43.885257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c883f1-be83-46d3-b5db-df92ccb09b0b None admin] RBAC: Authorizing `identity:get_group(group_id=895fc55542e14dbbaa927216b4e9442f)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:43.886704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c883f1-be83-46d3-b5db-df92ccb09b0b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:43.888398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 236/451] 10.222.0.60 () {56 vars in 1180 bytes} [Thu Jul 21 20:38:43 2022] GET /identity/v3/groups/895fc55542e14dbbaa927216b4e9442f => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:43.894231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c883f1-be83-46d3-b5db-df92ccb09b0b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:43.957678 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e2ef9-c87e-4a42-b6f7-6729421b7e54 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:43.992437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34022c-c676-4c5b-8108-f17ba54dab8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:43.996705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34022c-c676-4c5b-8108-f17ba54dab8d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:43.996705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34022c-c676-4c5b-8108-f17ba54dab8d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:43.996705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34022c-c676-4c5b-8108-f17ba54dab8d None admin] PATH_INFO: `/v3/projects/a9f8292e70024a6985791def05952106` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:43.996705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34022c-c676-4c5b-8108-f17ba54dab8d None admin] RBAC: Authorizing `identity:get_project(project_id=a9f8292e70024a6985791def05952106)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:43.997265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34022c-c676-4c5b-8108-f17ba54dab8d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:43.999005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 216/452] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:38:43 2022] GET /identity/v3/projects/a9f8292e70024a6985791def05952106 => generated 344 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:44.004480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34022c-c676-4c5b-8108-f17ba54dab8d None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:44.008600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c883f1-be83-46d3-b5db-df92ccb09b0b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:44.024119 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:44.044519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1990c-1a8a-4c19-8ae6-109db337e1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:44.045171 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1990c-1a8a-4c19-8ae6-109db337e1dd None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:44.045446 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1990c-1a8a-4c19-8ae6-109db337e1dd None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:44.045730 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1990c-1a8a-4c19-8ae6-109db337e1dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:44.046556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1990c-1a8a-4c19-8ae6-109db337e1dd None admin] RBAC: Adding query filter params (group.id=895fc55542e14dbbaa927216b4e9442f, role.id=04597f92ac654fb4ac68dbb783d3dc31, scope.project.id=3fa3d431d9cc468aafe7b895f047ff94) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:44.046867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1990c-1a8a-4c19-8ae6-109db337e1dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:44.048413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1990c-1a8a-4c19-8ae6-109db337e1dd None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:44.055385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 237/453] 10.222.0.60 () {56 vars in 1376 bytes} [Thu Jul 21 20:38:43 2022] GET /identity/v3/role_assignments?group.id=895fc55542e14dbbaa927216b4e9442f&scope.project.id=3fa3d431d9cc468aafe7b895f047ff94&role.id=04597f92ac654fb4ac68dbb783d3dc31 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:44.116828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34022c-c676-4c5b-8108-f17ba54dab8d None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:44.153571 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd22ce-de0f-4eab-b060-fe0c6a2c38bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:44.154444 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd22ce-de0f-4eab-b060-fe0c6a2c38bb None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:44.154723 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd22ce-de0f-4eab-b060-fe0c6a2c38bb None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:44.154995 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd22ce-de0f-4eab-b060-fe0c6a2c38bb None admin] PATH_INFO: `/v3/groups/c5825e2dfe954b378367ce6d292c71f8` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:44.156273 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd22ce-de0f-4eab-b060-fe0c6a2c38bb None admin] RBAC: Authorizing `identity:get_group(group_id=c5825e2dfe954b378367ce6d292c71f8)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:44.157725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd22ce-de0f-4eab-b060-fe0c6a2c38bb None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:44.159417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 217/454] 10.222.0.60 () {56 vars in 1180 bytes} [Thu Jul 21 20:38:44 2022] GET /identity/v3/groups/c5825e2dfe954b378367ce6d292c71f8 => generated 229 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:44.167221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1990c-1a8a-4c19-8ae6-109db337e1dd None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:44.232546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:44.279222 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1990c-1a8a-4c19-8ae6-109db337e1dd None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:44.316658 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93843b5c-b0c6-4df6-84de-b782be8daf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:44.317313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93843b5c-b0c6-4df6-84de-b782be8daf5c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:44.317596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93843b5c-b0c6-4df6-84de-b782be8daf5c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:44.317871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93843b5c-b0c6-4df6-84de-b782be8daf5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:44.318878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93843b5c-b0c6-4df6-84de-b782be8daf5c None admin] RBAC: Adding query filter params (group.id=c5825e2dfe954b378367ce6d292c71f8, role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=a9f8292e70024a6985791def05952106) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:44.319362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93843b5c-b0c6-4df6-84de-b782be8daf5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:44.320840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93843b5c-b0c6-4df6-84de-b782be8daf5c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:44.327139 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 238/455] 10.222.0.60 () {56 vars in 1376 bytes} [Thu Jul 21 20:38:44 2022] GET /identity/v3/role_assignments?group.id=c5825e2dfe954b378367ce6d292c71f8&scope.project.id=a9f8292e70024a6985791def05952106&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:47.012751 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca357bba-752e-4d10-8cbb-3da8d9102fa5 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:47.013271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca357bba-752e-4d10-8cbb-3da8d9102fa5 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:47.013271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca357bba-752e-4d10-8cbb-3da8d9102fa5 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:47.014894 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 218/456] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:47.020756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafe6dc-d22d-4a6b-99d2-16cf81e7bf1a None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:47.021677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafe6dc-d22d-4a6b-99d2-16cf81e7bf1a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:47.022350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbafe6dc-d22d-4a6b-99d2-16cf81e7bf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:47.045023 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-fbafe6dc-d22d-4a6b-99d2-16cf81e7bf1a None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:47.046602 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fbafe6dc-d22d-4a6b-99d2-16cf81e7bf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:47.136458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 239/457] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:38:47.180898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca357bba-752e-4d10-8cbb-3da8d9102fa5 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:47.192051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4c3e5-b1b5-4cd1-ab72-e5cc8c71d1c8 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:47.192051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4c3e5-b1b5-4cd1-ab72-e5cc8c71d1c8 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:47.192051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4c3e5-b1b5-4cd1-ab72-e5cc8c71d1c8 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:47.192788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 240/458] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:38:47.199549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bc647-f31f-4d13-aa22-9e46f165ea03 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:47.200158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bc647-f31f-4d13-aa22-9e46f165ea03 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:47.200633 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bc647-f31f-4d13-aa22-9e46f165ea03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:47.226611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-235bc647-f31f-4d13-aa22-9e46f165ea03 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:47.228421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-235bc647-f31f-4d13-aa22-9e46f165ea03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:47.299010 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:47.318746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 241/459] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:47.320367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5ce1d-d635-4126-8ee8-2a642cf471d9 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:47.320788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5ce1d-d635-4126-8ee8-2a642cf471d9 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:47.321223 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5ce1d-d635-4126-8ee8-2a642cf471d9 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:47.322878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 242/460] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:47.328505 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6b81e-9227-4054-8f7f-8a7e4c41bcf8 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:47.329001 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6b81e-9227-4054-8f7f-8a7e4c41bcf8 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:47.329427 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6b81e-9227-4054-8f7f-8a7e4c41bcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:47.336144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:47.357080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-44d6b81e-9227-4054-8f7f-8a7e4c41bcf8 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:47.358570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-44d6b81e-9227-4054-8f7f-8a7e4c41bcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:47.360043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca357bba-752e-4d10-8cbb-3da8d9102fa5 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:47.408375 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96dd628-520a-4122-b283-620eb616def1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:47.409450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96dd628-520a-4122-b283-620eb616def1 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:47.409888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96dd628-520a-4122-b283-620eb616def1 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:47.410365 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96dd628-520a-4122-b283-620eb616def1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:47.429246 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96dd628-520a-4122-b283-620eb616def1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:47.432143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96dd628-520a-4122-b283-620eb616def1 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:47.453441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 243/461] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:38:47.459737 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6b81e-9227-4054-8f7f-8a7e4c41bcf8 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:47.459906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-c96dd628-520a-4122-b283-620eb616def1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:47.459906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:47.459906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:47.459906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:47.459906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:47.459906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:47.459906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:47.459906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:47.459906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:47.459906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:47.459906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:47.459906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:47.459906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:47.459906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:47.459906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:47.460689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:47.461488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:47.461488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:47.461488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:47.461488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:47.463522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 219/462] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:47.464716 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96dd628-520a-4122-b283-620eb616def1 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:47.596397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96dd628-520a-4122-b283-620eb616def1 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:47.614051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6b81e-9227-4054-8f7f-8a7e4c41bcf8 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:47.635046 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:47.641482 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d7b9f-d0e7-426a-8199-11af094aa150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:47.642324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d7b9f-d0e7-426a-8199-11af094aa150 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:47.642830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d7b9f-d0e7-426a-8199-11af094aa150 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:47.643322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d7b9f-d0e7-426a-8199-11af094aa150 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:47.644385 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d7b9f-d0e7-426a-8199-11af094aa150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:47.644909 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d7b9f-d0e7-426a-8199-11af094aa150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:47.647163 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d7b9f-d0e7-426a-8199-11af094aa150 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:47.655299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:47.661475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196953a-e748-4cd8-89b5-e090875675f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:47.662181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 220/463] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:47.662431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196953a-e748-4cd8-89b5-e090875675f1 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:47.662855 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196953a-e748-4cd8-89b5-e090875675f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:47.663162 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196953a-e748-4cd8-89b5-e090875675f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:47.663930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d7b9f-d0e7-426a-8199-11af094aa150 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:47.679949 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196953a-e748-4cd8-89b5-e090875675f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:47.681830 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196953a-e748-4cd8-89b5-e090875675f1 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:47.702396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-8196953a-e748-4cd8-89b5-e090875675f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:47.702396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:47.702396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:47.702396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:47.702396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:47.702396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:47.702396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:47.702396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:47.702396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:47.702396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:47.702396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:47.702396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:47.702396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:47.702396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:47.702396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:47.703295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:47.704237 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:47.704237 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:47.704237 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:47.704237 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:47.706380 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 244/464] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:47.707390 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196953a-e748-4cd8-89b5-e090875675f1 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:47.811235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d7b9f-d0e7-426a-8199-11af094aa150 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:47.839929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196953a-e748-4cd8-89b5-e090875675f1 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:47.856598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de9251-ebcc-4d6f-922c-c02c37db29ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:47.857788 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de9251-ebcc-4d6f-922c-c02c37db29ac None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:47.858057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de9251-ebcc-4d6f-922c-c02c37db29ac None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:47.858291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de9251-ebcc-4d6f-922c-c02c37db29ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:47.870991 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de9251-ebcc-4d6f-922c-c02c37db29ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:47.872507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de9251-ebcc-4d6f-922c-c02c37db29ac None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:47.878631 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ec3c0-201a-4e5a-a82e-3875c28ef9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:47.879887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ec3c0-201a-4e5a-a82e-3875c28ef9df None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:47.880182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ec3c0-201a-4e5a-a82e-3875c28ef9df None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:47.880464 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ec3c0-201a-4e5a-a82e-3875c28ef9df None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:47.881861 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ec3c0-201a-4e5a-a82e-3875c28ef9df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:47.883405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ec3c0-201a-4e5a-a82e-3875c28ef9df None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:47.890714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-21de9251-ebcc-4d6f-922c-c02c37db29ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:47.890714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:47.890714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:47.890714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:47.890714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:47.890714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:47.890714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:47.890714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:47.890714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:47.890714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:47.890714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:47.890714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:47.890714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:47.890714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:47.890714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:47.891541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:47.892408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:47.892408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:47.892408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:47.892408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:47.893490 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 221/465] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:47.894047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de9251-ebcc-4d6f-922c-c02c37db29ac None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:47.911405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 245/466] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:38:47 2022] POST /identity/v3/users => generated 246 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:38:47.913425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ec47c-9e50-41bf-b9b9-5b1e869a05ea None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:47.913739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ec47c-9e50-41bf-b9b9-5b1e869a05ea None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:47.913989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ec47c-9e50-41bf-b9b9-5b1e869a05ea None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:47.915510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 246/467] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:47.920472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ec47c-9e50-41bf-b9b9-5b1e869a05ea None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:47.956249 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:47.980602 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:48.058694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de9251-ebcc-4d6f-922c-c02c37db29ac None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:48.079863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ec47c-9e50-41bf-b9b9-5b1e869a05ea None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:48.108659 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76dad6-c943-42b7-bd8e-63fc96817cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:48.109257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76dad6-c943-42b7-bd8e-63fc96817cb5 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:48.109544 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76dad6-c943-42b7-bd8e-63fc96817cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:48.109803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76dad6-c943-42b7-bd8e-63fc96817cb5 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:48.110529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76dad6-c943-42b7-bd8e-63fc96817cb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:48.110837 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76dad6-c943-42b7-bd8e-63fc96817cb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:48.113153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c76dad6-c943-42b7-bd8e-63fc96817cb5 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:48.126881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a9232-124e-4457-9fbe-c369f76c4fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:48.127448 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 222/468] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:48.127555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a9232-124e-4457-9fbe-c369f76c4fb3 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:48.128751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a9232-124e-4457-9fbe-c369f76c4fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:48.128751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a9232-124e-4457-9fbe-c369f76c4fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:48.128932 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146efa6c-24fb-48c6-b8d6-b60e570cd3db None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:48.129002 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a9232-124e-4457-9fbe-c369f76c4fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:48.129192 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146efa6c-24fb-48c6-b8d6-b60e570cd3db None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:48.129290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a9232-124e-4457-9fbe-c369f76c4fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:48.129360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146efa6c-24fb-48c6-b8d6-b60e570cd3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:48.130708 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a9232-124e-4457-9fbe-c369f76c4fb3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:48.150704 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-146efa6c-24fb-48c6-b8d6-b60e570cd3db None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:48.151750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 247/469] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:48.156543 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-146efa6c-24fb-48c6-b8d6-b60e570cd3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:48.156712 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a9232-124e-4457-9fbe-c369f76c4fb3 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:48.249288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 223/470] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:48.250801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146efa6c-24fb-48c6-b8d6-b60e570cd3db None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:48.277586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:48.298901 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a9232-124e-4457-9fbe-c369f76c4fb3 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:48.308165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:48.338107 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26241cbe-8bad-4d72-acbc-1e0502adc65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:48.338918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26241cbe-8bad-4d72-acbc-1e0502adc65e None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:48.339202 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26241cbe-8bad-4d72-acbc-1e0502adc65e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:48.339522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26241cbe-8bad-4d72-acbc-1e0502adc65e None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:48.340275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26241cbe-8bad-4d72-acbc-1e0502adc65e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:48.341800 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26241cbe-8bad-4d72-acbc-1e0502adc65e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:48.367103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 248/471] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:38:48 2022] POST /identity/v3/users => generated 248 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:38:48.368003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26241cbe-8bad-4d72-acbc-1e0502adc65e None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:48.385429 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146efa6c-24fb-48c6-b8d6-b60e570cd3db None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:48.448541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd9329-d366-4958-a450-6dcb93a50e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:48.449658 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd9329-d366-4958-a450-6dcb93a50e2d None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:48.450123 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd9329-d366-4958-a450-6dcb93a50e2d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:48.450562 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd9329-d366-4958-a450-6dcb93a50e2d None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:48.451754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd9329-d366-4958-a450-6dcb93a50e2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:48.457165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd9329-d366-4958-a450-6dcb93a50e2d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:48.494061 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 224/472] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:38:48 2022] POST /identity/v3/users => generated 248 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:38:48.496008 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c865f68-eb3e-4fb3-9497-fe3a71e257ec None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:48.496628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c865f68-eb3e-4fb3-9497-fe3a71e257ec None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:48.497119 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c865f68-eb3e-4fb3-9497-fe3a71e257ec None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:48.503659 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 225/473] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:48.506717 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2d1fa-7b6a-4760-9ce5-49872f862406 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:48.507469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2d1fa-7b6a-4760-9ce5-49872f862406 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:48.508074 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2d1fa-7b6a-4760-9ce5-49872f862406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:48.528994 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-41f2d1fa-7b6a-4760-9ce5-49872f862406 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:48.530789 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-41f2d1fa-7b6a-4760-9ce5-49872f862406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:48.536226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26241cbe-8bad-4d72-acbc-1e0502adc65e None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:48.584849 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc72fa-bc2c-4ac4-8d19-f8fd86ab9b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:48.585736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc72fa-bc2c-4ac4-8d19-f8fd86ab9b2c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:48.586027 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc72fa-bc2c-4ac4-8d19-f8fd86ab9b2c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:48.586309 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc72fa-bc2c-4ac4-8d19-f8fd86ab9b2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:48.598706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc72fa-bc2c-4ac4-8d19-f8fd86ab9b2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:48.600796 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc72fa-bc2c-4ac4-8d19-f8fd86ab9b2c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:48.605132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:48.621377 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 226/474] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:38:48.630350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-4bfc72fa-bc2c-4ac4-8d19-f8fd86ab9b2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:48.630350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:48.630350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:48.630350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:48.630350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:48.630350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:48.630350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:48.630350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:48.630350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:48.630350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:48.630350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:48.630350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:48.630350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:48.630350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:48.630350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:48.632040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:48.632971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:48.632971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:48.632971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:48.632971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:48.632971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 249/475] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:48.639131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2d1fa-7b6a-4760-9ce5-49872f862406 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:48.650436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc72fa-bc2c-4ac4-8d19-f8fd86ab9b2c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:48.676011 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:48.780092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc72fa-bc2c-4ac4-8d19-f8fd86ab9b2c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:48.780453 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2d1fa-7b6a-4760-9ce5-49872f862406 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:48.835739 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfabf98-d882-47d4-ba34-f6bcd1f33db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:48.836077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b21b78-0e1d-4a15-b37d-a79964d9a773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:48.836444 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfabf98-d882-47d4-ba34-f6bcd1f33db0 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:48.836727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfabf98-d882-47d4-ba34-f6bcd1f33db0 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:48.836831 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b21b78-0e1d-4a15-b37d-a79964d9a773 None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:48.836999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfabf98-d882-47d4-ba34-f6bcd1f33db0 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:48.837097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b21b78-0e1d-4a15-b37d-a79964d9a773 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:48.837275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b21b78-0e1d-4a15-b37d-a79964d9a773 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:48.837742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfabf98-d882-47d4-ba34-f6bcd1f33db0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:48.838046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfabf98-d882-47d4-ba34-f6bcd1f33db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:48.838151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b21b78-0e1d-4a15-b37d-a79964d9a773 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:48.839464 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfabf98-d882-47d4-ba34-f6bcd1f33db0 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:48.839592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b21b78-0e1d-4a15-b37d-a79964d9a773 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:48.848587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.sql.core [None req-f8b21b78-0e1d-4a15-b37d-a79964d9a773 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 20:38:48.848587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 20:38:48.848587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [parameters: {'id': 'b2c31e233fcf4e6d976cadff61810a16', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 20:38:48.848587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87061) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 20:38:48.856377 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 227/476] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:48.861606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfabf98-d882-47d4-ba34-f6bcd1f33db0 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:48.870689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-f8b21b78-0e1d-4a15-b37d-a79964d9a773 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 20:38:48.870689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:48.870689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 20:38:48.870689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 20:38:48.870689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 20:38:48.870689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 20:38:48.870689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 20:38:48.870689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 20:38:48.870689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 20:38:48.870689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application conn.query(q) Jul 21 20:38:48.870689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 20:38:48.870689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 20:38:48.870689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 20:38:48.870689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application result.read() Jul 21 20:38:48.870689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return ref.to_dict() Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 20:38:48.871701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application next(self.gen) Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application yield resource Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application next(self.gen) Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise self.value Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application session.commit() Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 20:38:48.873155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self.session.flush() Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application compat.raise_( Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 20:38:48.874053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 20:38:48.875125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application conn.query(q) Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application result.read() Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 20:38:48.876082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [parameters: {'id': 'b2c31e233fcf4e6d976cadff61810a16', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:48.876921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 20:38:48.877791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:48.878742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 250/477] 10.222.0.60 () {60 vars in 1131 bytes} [Thu Jul 21 20:38:48 2022] POST /identity/v3/roles => generated 144 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:38:48.886571 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b21b78-0e1d-4a15-b37d-a79964d9a773 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:48.978571 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:48.982184 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:49.002962 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfabf98-d882-47d4-ba34-f6bcd1f33db0 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:49.040547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b21b78-0e1d-4a15-b37d-a79964d9a773 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:49.049768 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b2401-54a5-4a15-92c0-029732d0d224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:49.050462 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b2401-54a5-4a15-92c0-029732d0d224 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:49.050748 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b2401-54a5-4a15-92c0-029732d0d224 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:49.051732 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b2401-54a5-4a15-92c0-029732d0d224 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:49.052495 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b2401-54a5-4a15-92c0-029732d0d224 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:49.055301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b2401-54a5-4a15-92c0-029732d0d224 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:49.088407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ef624-eb5a-48db-9f74-be8283e79b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:49.089387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ef624-eb5a-48db-9f74-be8283e79b0d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:49.089672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ef624-eb5a-48db-9f74-be8283e79b0d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:49.090006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ef624-eb5a-48db-9f74-be8283e79b0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:49.100035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 228/478] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:38:48 2022] POST /identity/v3/users => generated 249 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:38:49.102198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ef624-eb5a-48db-9f74-be8283e79b0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-6d4ef624-eb5a-48db-9f74-be8283e79b0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:49.111242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:49.113550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:49.116189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:49.116189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:49.116189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:49.116189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:49.116189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:49.116189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:49.116189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:49.116189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:49.116189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:49.116189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:49.116189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 20:38:49.116189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:49.116189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 251/479] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:38:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:49.124883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b2401-54a5-4a15-92c0-029732d0d224 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:49.283926 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b2401-54a5-4a15-92c0-029732d0d224 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:49.330227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d582774-ade0-424f-b2bc-f2a5db76e7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:49.331354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d582774-ade0-424f-b2bc-f2a5db76e7f4 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:49.331896 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d582774-ade0-424f-b2bc-f2a5db76e7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:49.332374 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d582774-ade0-424f-b2bc-f2a5db76e7f4 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:49.333342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d582774-ade0-424f-b2bc-f2a5db76e7f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:49.333876 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d582774-ade0-424f-b2bc-f2a5db76e7f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:49.335688 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d582774-ade0-424f-b2bc-f2a5db76e7f4 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:49.351901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 229/480] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:50.527106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cc694-0603-4eaf-8890-688a62f8ae2b None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:50.528171 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cc694-0603-4eaf-8890-688a62f8ae2b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:50.528671 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cc694-0603-4eaf-8890-688a62f8ae2b None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:50.530527 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 252/481] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:50.536623 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c026e-5519-4a79-b43d-e33508e4d76b None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:50.537978 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c026e-5519-4a79-b43d-e33508e4d76b None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:50.538526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c026e-5519-4a79-b43d-e33508e4d76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:50.547914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:50.571063 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-292c026e-5519-4a79-b43d-e33508e4d76b None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:50.572302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-292c026e-5519-4a79-b43d-e33508e4d76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:50.674122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 230/482] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:50.712613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cc694-0603-4eaf-8890-688a62f8ae2b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:50.848387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cc694-0603-4eaf-8890-688a62f8ae2b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:50.865343 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:50.887569 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280642c-e338-49bd-8c36-d1ba0a1bcc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:50.888402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280642c-e338-49bd-8c36-d1ba0a1bcc1f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:50.888680 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280642c-e338-49bd-8c36-d1ba0a1bcc1f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:50.888959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280642c-e338-49bd-8c36-d1ba0a1bcc1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:50.905463 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280642c-e338-49bd-8c36-d1ba0a1bcc1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-2280642c-e338-49bd-8c36-d1ba0a1bcc1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:50.914238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:50.915603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:50.916513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:50.916513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:50.916513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:50.916513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:50.916513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:50.916513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:50.916513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:50.916513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:50.916513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:50.916513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:50.916513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:38:50.916513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:50.916513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 253/483] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:38:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:50.925279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c026e-5519-4a79-b43d-e33508e4d76b None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:51.100497 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c026e-5519-4a79-b43d-e33508e4d76b None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:51.154317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ed803-4dd2-47a3-8e6c-1506f30194ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:51.155006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ed803-4dd2-47a3-8e6c-1506f30194ae None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:51.155295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ed803-4dd2-47a3-8e6c-1506f30194ae None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:51.155687 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ed803-4dd2-47a3-8e6c-1506f30194ae None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:51.156487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ed803-4dd2-47a3-8e6c-1506f30194ae None admin] RBAC: Adding query filter params (name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:51.156795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ed803-4dd2-47a3-8e6c-1506f30194ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:51.158185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ed803-4dd2-47a3-8e6c-1506f30194ae None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:51.159724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:51.176188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 231/484] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:38:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:51.177352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280642c-e338-49bd-8c36-d1ba0a1bcc1f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:51.337305 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280642c-e338-49bd-8c36-d1ba0a1bcc1f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:51.393312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da779a3-9806-42b5-b3b4-3cc8c3e35b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:51.394108 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da779a3-9806-42b5-b3b4-3cc8c3e35b0f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:51.394387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da779a3-9806-42b5-b3b4-3cc8c3e35b0f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:51.394661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da779a3-9806-42b5-b3b4-3cc8c3e35b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:51.412517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da779a3-9806-42b5-b3b4-3cc8c3e35b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:51.416733 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da779a3-9806-42b5-b3b4-3cc8c3e35b0f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:51.418241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045a587-2e14-4cda-b71b-005ed03a464d None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:51.418241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045a587-2e14-4cda-b71b-005ed03a464d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:51.418241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045a587-2e14-4cda-b71b-005ed03a464d None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:51.419402 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 232/485] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:51.425087 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6293c-5a2a-490a-b9ea-8e783102477b None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:51.425369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6293c-5a2a-490a-b9ea-8e783102477b None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:51.425646 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6293c-5a2a-490a-b9ea-8e783102477b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:51.445702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-14d6293c-5a2a-490a-b9ea-8e783102477b None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:51.446808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-14d6293c-5a2a-490a-b9ea-8e783102477b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:51.447896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-3da779a3-9806-42b5-b3b4-3cc8c3e35b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:51.447896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:51.447896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:51.447896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:51.447896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:51.447896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:51.447896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:51.447896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:51.447896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:51.447896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:51.447896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:51.447896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:51.447896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:51.447896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:51.447896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:51.448766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:51.450386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:51.450386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:51.450386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:51.450386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:51.451813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 254/486] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:51.458007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da779a3-9806-42b5-b3b4-3cc8c3e35b0f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:51.500110 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:51.538043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 233/487] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:51.539879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf72270-f7b6-4c8a-a3b5-5984e702777a None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:51.540507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf72270-f7b6-4c8a-a3b5-5984e702777a None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:51.540790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf72270-f7b6-4c8a-a3b5-5984e702777a None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:51.547148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 234/488] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:51.549326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fd00e-b74d-434b-a38f-8bed0b23b568 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:51.549611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fd00e-b74d-434b-a38f-8bed0b23b568 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:51.549889 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fd00e-b74d-434b-a38f-8bed0b23b568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:51.571037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-df5fd00e-b74d-434b-a38f-8bed0b23b568 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:51.572810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-df5fd00e-b74d-434b-a38f-8bed0b23b568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:51.607555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da779a3-9806-42b5-b3b4-3cc8c3e35b0f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:51.649987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828d78a-c7d2-4acc-a66d-b0aae2c15e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:51.650661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828d78a-c7d2-4acc-a66d-b0aae2c15e9f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:51.651063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828d78a-c7d2-4acc-a66d-b0aae2c15e9f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:51.651340 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828d78a-c7d2-4acc-a66d-b0aae2c15e9f None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:51.652215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828d78a-c7d2-4acc-a66d-b0aae2c15e9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:51.652521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828d78a-c7d2-4acc-a66d-b0aae2c15e9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:51.654009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828d78a-c7d2-4acc-a66d-b0aae2c15e9f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:51.671914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 235/489] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:51.671914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fd00e-b74d-434b-a38f-8bed0b23b568 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:51.676483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 255/490] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:51.681884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828d78a-c7d2-4acc-a66d-b0aae2c15e9f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:51.763486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:51.794631 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:51.812914 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828d78a-c7d2-4acc-a66d-b0aae2c15e9f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:51.825449 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fd00e-b74d-434b-a38f-8bed0b23b568 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:51.852209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c6237-0ccd-4a16-bfa6-807b373efba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:51.853314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c6237-0ccd-4a16-bfa6-807b373efba3 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:51.853802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c6237-0ccd-4a16-bfa6-807b373efba3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:51.854511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c6237-0ccd-4a16-bfa6-807b373efba3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:51.862918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c6237-0ccd-4a16-bfa6-807b373efba3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:51.864689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c6237-0ccd-4a16-bfa6-807b373efba3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:51.871447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd62a5-b575-4c88-887e-731ec20caa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:51.872492 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd62a5-b575-4c88-887e-731ec20caa46 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:51.872781 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd62a5-b575-4c88-887e-731ec20caa46 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:51.873059 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd62a5-b575-4c88-887e-731ec20caa46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-c78c6237-0ccd-4a16-bfa6-807b373efba3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:38:51.883787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:51.884762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:51.885640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:51.885640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:51.885640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:38:51.885640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:38:51.885640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:38:51.885640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:38:51.885640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:38:51.885640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:38:51.885640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:38:51.885640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:51.886146 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd62a5-b575-4c88-887e-731ec20caa46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:51.888467 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 256/491] 10.222.0.60 () {56 vars in 1093 bytes} [Thu Jul 21 20:38:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:51.889630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c6237-0ccd-4a16-bfa6-807b373efba3 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-78fd62a5-b575-4c88-887e-731ec20caa46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:51.893342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:51.894225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:51.895394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:51.895394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:51.895394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:51.895394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:51.895394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:51.895394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:51.895394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:51.895394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:51.895394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:51.895394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:51.895394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:38:51.895394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:51.895394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 236/492] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:38:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:51.895394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd62a5-b575-4c88-887e-731ec20caa46 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:52.017732 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd62a5-b575-4c88-887e-731ec20caa46 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:52.019322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c6237-0ccd-4a16-bfa6-807b373efba3 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:52.035444 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:52.056414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0adaa-6790-4c48-850e-dca785c5fbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:52.057151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0adaa-6790-4c48-850e-dca785c5fbbc None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:52.057435 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0adaa-6790-4c48-850e-dca785c5fbbc None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:52.057713 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0adaa-6790-4c48-850e-dca785c5fbbc None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:52.058563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0adaa-6790-4c48-850e-dca785c5fbbc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:52.058871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0adaa-6790-4c48-850e-dca785c5fbbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:52.059053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dfd54-ea8d-4f79-856f-17b81d47b79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:52.059878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dfd54-ea8d-4f79-856f-17b81d47b79b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:52.060159 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dfd54-ea8d-4f79-856f-17b81d47b79b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:52.060344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0adaa-6790-4c48-850e-dca785c5fbbc None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:52.060446 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dfd54-ea8d-4f79-856f-17b81d47b79b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:52.074966 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65dfd54-ea8d-4f79-856f-17b81d47b79b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-f65dfd54-ea8d-4f79-856f-17b81d47b79b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:52.083354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:52.084731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:52.085586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:52.085586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:52.085586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:52.085586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:52.085586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:52.085586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:52.085586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:52.085586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:52.085586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:52.085586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:52.085586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:38:52.085586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:52.085586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 257/493] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:38:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:52.086914 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dfd54-ea8d-4f79-856f-17b81d47b79b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:52.096300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:52.106884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 237/494] 10.222.0.60 () {56 vars in 1133 bytes} [Thu Jul 21 20:38:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:52.108596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b9df8-16f5-49e8-9ab3-aef1b774e080 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:52.109001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b9df8-16f5-49e8-9ab3-aef1b774e080 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:52.109290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b9df8-16f5-49e8-9ab3-aef1b774e080 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:52.111753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 238/495] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:52.112035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199cf65-dbfd-411b-a7c7-5a74b79f55cb None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:52.112307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199cf65-dbfd-411b-a7c7-5a74b79f55cb None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:52.112592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199cf65-dbfd-411b-a7c7-5a74b79f55cb None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:52.116094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 239/496] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:52.116094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199cf65-dbfd-411b-a7c7-5a74b79f55cb None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:52.193731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dfd54-ea8d-4f79-856f-17b81d47b79b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:52.221117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199cf65-dbfd-411b-a7c7-5a74b79f55cb None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:52.241398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec00ef1-cf77-4ee7-8b4e-502d4bd7e303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:52.242250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec00ef1-cf77-4ee7-8b4e-502d4bd7e303 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:52.242543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec00ef1-cf77-4ee7-8b4e-502d4bd7e303 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:52.242826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec00ef1-cf77-4ee7-8b4e-502d4bd7e303 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:52.243593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec00ef1-cf77-4ee7-8b4e-502d4bd7e303 None admin] RBAC: Adding query filter params (name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:52.243942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec00ef1-cf77-4ee7-8b4e-502d4bd7e303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:52.245351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec00ef1-cf77-4ee7-8b4e-502d4bd7e303 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:52.261014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 258/497] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:38:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:52.261225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755b25c-8127-4a6c-b2c2-c4586f45e2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:52.261393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec00ef1-cf77-4ee7-8b4e-502d4bd7e303 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:52.261537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755b25c-8127-4a6c-b2c2-c4586f45e2d6 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:52.261537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755b25c-8127-4a6c-b2c2-c4586f45e2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:52.261537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755b25c-8127-4a6c-b2c2-c4586f45e2d6 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:52.261537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755b25c-8127-4a6c-b2c2-c4586f45e2d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:52.261537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755b25c-8127-4a6c-b2c2-c4586f45e2d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:52.261537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755b25c-8127-4a6c-b2c2-c4586f45e2d6 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:52.269517 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 240/498] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:38:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:52.270762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2dc76-e6ff-4d90-ae70-cf910b08d1f5 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:52.271037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2dc76-e6ff-4d90-ae70-cf910b08d1f5 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:52.271310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2dc76-e6ff-4d90-ae70-cf910b08d1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:52.288280 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-6fe2dc76-e6ff-4d90-ae70-cf910b08d1f5 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:52.289496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6fe2dc76-e6ff-4d90-ae70-cf910b08d1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:52.319020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:52.335351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:52.360743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 241/499] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:52.362027 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcd20a-b1b6-4096-9d6d-fac1052c1f3c None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:52.365353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcd20a-b1b6-4096-9d6d-fac1052c1f3c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:52.365353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcd20a-b1b6-4096-9d6d-fac1052c1f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:52.380217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec00ef1-cf77-4ee7-8b4e-502d4bd7e303 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:52.380726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-00bcd20a-b1b6-4096-9d6d-fac1052c1f3c None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:52.382095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-00bcd20a-b1b6-4096-9d6d-fac1052c1f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:52.422649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3039be-1c33-4edf-9d18-ca47407a2557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:52.423305 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3039be-1c33-4edf-9d18-ca47407a2557 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:52.423588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3039be-1c33-4edf-9d18-ca47407a2557 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:52.423898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3039be-1c33-4edf-9d18-ca47407a2557 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:52.462856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 242/500] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:52.462856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcd20a-b1b6-4096-9d6d-fac1052c1f3c None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:52.506731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3039be-1c33-4edf-9d18-ca47407a2557 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:52.508397 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3039be-1c33-4edf-9d18-ca47407a2557 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:52.579852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:52.589899 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:52.590274 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcd20a-b1b6-4096-9d6d-fac1052c1f3c None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:52.637236 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 259/501] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:52.638082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3039be-1c33-4edf-9d18-ca47407a2557 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:52.638569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d9ca3-fd11-4020-937e-5cbc50bbd592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:52.639383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d9ca3-fd11-4020-937e-5cbc50bbd592 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:52.639669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d9ca3-fd11-4020-937e-5cbc50bbd592 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:52.639976 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d9ca3-fd11-4020-937e-5cbc50bbd592 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:52.651936 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d9ca3-fd11-4020-937e-5cbc50bbd592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:52.653430 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d9ca3-fd11-4020-937e-5cbc50bbd592 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:52.672042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-1a8d9ca3-fd11-4020-937e-5cbc50bbd592 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:52.672042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:52.672042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:52.672042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:52.672042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:52.672042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:52.672042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:52.672042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:52.672042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:52.672042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:52.672042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:52.672042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:52.672042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:52.672042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:52.672042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:52.673088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:52.673925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:52.673925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:52.673925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:52.673925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:52.673925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 243/502] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:52.674174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d9ca3-fd11-4020-937e-5cbc50bbd592 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:52.747350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3039be-1c33-4edf-9d18-ca47407a2557 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:52.781804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59003225-ccf3-4ee0-bb50-34f640b2a03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:52.782593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59003225-ccf3-4ee0-bb50-34f640b2a03b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:52.782871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59003225-ccf3-4ee0-bb50-34f640b2a03b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:52.783142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59003225-ccf3-4ee0-bb50-34f640b2a03b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:52.785081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d9ca3-fd11-4020-937e-5cbc50bbd592 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:52.794404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59003225-ccf3-4ee0-bb50-34f640b2a03b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:52.795885 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59003225-ccf3-4ee0-bb50-34f640b2a03b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:52.814244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-59003225-ccf3-4ee0-bb50-34f640b2a03b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:52.814244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:52.814244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:52.814244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:52.814244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:52.814244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:52.814244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:52.814244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:52.814244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:52.814244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:52.814244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:52.814244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:52.814244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:52.814244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:52.814244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:52.815260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:52.816876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:52.816876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:52.816876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:52.816876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:52.816876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 260/503] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:52.816876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59003225-ccf3-4ee0-bb50-34f640b2a03b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:52.825685 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e3dc5-aba9-4f39-a0ae-9038db133d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:52.827032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3dc5-aba9-4f39-a0ae-9038db133d4c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:52.827500 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3dc5-aba9-4f39-a0ae-9038db133d4c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:52.828042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3dc5-aba9-4f39-a0ae-9038db133d4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:52.841184 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:52.844566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e3dc5-aba9-4f39-a0ae-9038db133d4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-720e3dc5-aba9-4f39-a0ae-9038db133d4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:52.853452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:52.854639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:52.855400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:52.855400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:52.855400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:52.855400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:52.855400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:52.855400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:52.855400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:52.855400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:52.855400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:52.855400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:52.855400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:38:52.855400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:52.855400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 244/504] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:38:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:52.856413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e3dc5-aba9-4f39-a0ae-9038db133d4c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:52.872547 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:52.931022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59003225-ccf3-4ee0-bb50-34f640b2a03b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:52.969750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e3dc5-aba9-4f39-a0ae-9038db133d4c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:52.970112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac838786-b67c-421e-afc7-b3731c88163a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:52.970917 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac838786-b67c-421e-afc7-b3731c88163a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:52.971213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac838786-b67c-421e-afc7-b3731c88163a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:52.971492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac838786-b67c-421e-afc7-b3731c88163a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:52.982794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac838786-b67c-421e-afc7-b3731c88163a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:52.984294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac838786-b67c-421e-afc7-b3731c88163a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:53.002401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-ac838786-b67c-421e-afc7-b3731c88163a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:53.002401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:53.002401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:53.002401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:53.002401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:53.002401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:53.002401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:53.002401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:53.002401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:53.002401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:53.002401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:53.002401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:53.002401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:53.002401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:53.002401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:53.003310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:53.004516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:53.004516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:53.004516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:53.004516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:53.004516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 261/505] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:53.004516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac838786-b67c-421e-afc7-b3731c88163a None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:53.011941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccb44a-da63-46e6-9d39-4f13ea3d60d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:53.012748 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccb44a-da63-46e6-9d39-4f13ea3d60d6 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:53.013030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccb44a-da63-46e6-9d39-4f13ea3d60d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:53.013304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccb44a-da63-46e6-9d39-4f13ea3d60d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:53.024913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccb44a-da63-46e6-9d39-4f13ea3d60d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:53.026395 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccb44a-da63-46e6-9d39-4f13ea3d60d6 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:53.044042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-e6ccb44a-da63-46e6-9d39-4f13ea3d60d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:53.044042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:53.044042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:53.044042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:53.044042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:53.044042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:53.044042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:53.044042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:53.044042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:53.044042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:53.044042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:53.044042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:53.044042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:53.044042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:53.044042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:53.044931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:53.045764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:53.045764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:53.045764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:53.045764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:53.045764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 245/506] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:53.046121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccb44a-da63-46e6-9d39-4f13ea3d60d6 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:53.086647 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:53.113115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:53.121138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac838786-b67c-421e-afc7-b3731c88163a None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:53.157165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccb44a-da63-46e6-9d39-4f13ea3d60d6 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:53.163601 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58a5e6-70d0-49fd-a40f-2f8d80a1b6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:53.164296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58a5e6-70d0-49fd-a40f-2f8d80a1b6d6 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:53.165009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58a5e6-70d0-49fd-a40f-2f8d80a1b6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:53.165311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58a5e6-70d0-49fd-a40f-2f8d80a1b6d6 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:53.166044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58a5e6-70d0-49fd-a40f-2f8d80a1b6d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:53.166367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58a5e6-70d0-49fd-a40f-2f8d80a1b6d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:53.167791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58a5e6-70d0-49fd-a40f-2f8d80a1b6d6 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:53.184156 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 262/507] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:53.184156 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58a5e6-70d0-49fd-a40f-2f8d80a1b6d6 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:53.199805 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c789de5-be36-401e-a252-b7291994d05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:53.200479 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c789de5-be36-401e-a252-b7291994d05e None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:53.200759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c789de5-be36-401e-a252-b7291994d05e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:53.201032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c789de5-be36-401e-a252-b7291994d05e None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:53.201840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c789de5-be36-401e-a252-b7291994d05e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:53.202143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c789de5-be36-401e-a252-b7291994d05e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:53.203554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c789de5-be36-401e-a252-b7291994d05e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:53.217225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 246/508] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:53.218059 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c789de5-be36-401e-a252-b7291994d05e None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:53.296335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58a5e6-70d0-49fd-a40f-2f8d80a1b6d6 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:53.323986 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c789de5-be36-401e-a252-b7291994d05e None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:53.326470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:53.332045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfed59-5907-426b-a960-630e44040df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:53.332722 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfed59-5907-426b-a960-630e44040df9 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:53.333005 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfed59-5907-426b-a960-630e44040df9 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:53.333282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfed59-5907-426b-a960-630e44040df9 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:53.334879 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfed59-5907-426b-a960-630e44040df9 None admin] RBAC: Adding query filter params (name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:53.335328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfed59-5907-426b-a960-630e44040df9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:53.336743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfed59-5907-426b-a960-630e44040df9 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:53.345122 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 263/509] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:38:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:53.345992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfed59-5907-426b-a960-630e44040df9 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:53.352541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:53.364996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9498bd-7762-4c6b-86ad-9420012da5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:53.365632 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9498bd-7762-4c6b-86ad-9420012da5b7 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:53.365916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9498bd-7762-4c6b-86ad-9420012da5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:53.366191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9498bd-7762-4c6b-86ad-9420012da5b7 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:53.366919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9498bd-7762-4c6b-86ad-9420012da5b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:53.367489 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9498bd-7762-4c6b-86ad-9420012da5b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:53.368655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9498bd-7762-4c6b-86ad-9420012da5b7 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:53.380616 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 247/510] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:53.381487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9498bd-7762-4c6b-86ad-9420012da5b7 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:53.462176 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfed59-5907-426b-a960-630e44040df9 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:53.497954 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf7e58-86c1-410b-a499-adb78632fca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:53.498611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf7e58-86c1-410b-a499-adb78632fca9 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:53.498891 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf7e58-86c1-410b-a499-adb78632fca9 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:53.499166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf7e58-86c1-410b-a499-adb78632fca9 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:53.499732 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9498bd-7762-4c6b-86ad-9420012da5b7 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:53.499938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf7e58-86c1-410b-a499-adb78632fca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:53.500273 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf7e58-86c1-410b-a499-adb78632fca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:53.501593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf7e58-86c1-410b-a499-adb78632fca9 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:53.515164 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 264/511] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:53.516048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf7e58-86c1-410b-a499-adb78632fca9 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:53.535665 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eac51a-33fd-4fa1-8708-dbf4d24a9575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:53.536497 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eac51a-33fd-4fa1-8708-dbf4d24a9575 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:53.536777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eac51a-33fd-4fa1-8708-dbf4d24a9575 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:53.537098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eac51a-33fd-4fa1-8708-dbf4d24a9575 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:53.546379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eac51a-33fd-4fa1-8708-dbf4d24a9575 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:53.547859 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eac51a-33fd-4fa1-8708-dbf4d24a9575 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-04eac51a-33fd-4fa1-8708-dbf4d24a9575 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:38:53.564265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:53.565229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:53.566095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:53.566095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:53.566095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:38:53.566095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:38:53.566095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:38:53.566095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:38:53.566095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:38:53.566095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:38:53.566095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:38:53.566095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:53.566095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 248/512] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:53.566673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eac51a-33fd-4fa1-8708-dbf4d24a9575 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:53.578129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:53.612728 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:53.628104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf7e58-86c1-410b-a499-adb78632fca9 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:53.665129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed33796-d1eb-46ad-8d37-c0c844ea8711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:53.666099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed33796-d1eb-46ad-8d37-c0c844ea8711 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:53.666216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed33796-d1eb-46ad-8d37-c0c844ea8711 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:53.666481 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed33796-d1eb-46ad-8d37-c0c844ea8711 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:53.674216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed33796-d1eb-46ad-8d37-c0c844ea8711 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:53.675677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed33796-d1eb-46ad-8d37-c0c844ea8711 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:53.676545 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eac51a-33fd-4fa1-8708-dbf4d24a9575 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-5ed33796-d1eb-46ad-8d37-c0c844ea8711 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:38:53.690718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:53.691665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:53.692577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:53.692577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:53.692577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:38:53.692577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:38:53.692577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:38:53.692577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:38:53.692577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:38:53.692577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:38:53.692577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 20:38:53.692577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:53.692577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 265/513] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:38:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:53.693127 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed33796-d1eb-46ad-8d37-c0c844ea8711 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:53.718910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd13d76-56c4-4210-9f93-af40d5a5ca56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:53.719703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd13d76-56c4-4210-9f93-af40d5a5ca56 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:53.720008 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd13d76-56c4-4210-9f93-af40d5a5ca56 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:53.720280 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd13d76-56c4-4210-9f93-af40d5a5ca56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:53.731486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd13d76-56c4-4210-9f93-af40d5a5ca56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:53.733004 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd13d76-56c4-4210-9f93-af40d5a5ca56 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:53.750724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-1fd13d76-56c4-4210-9f93-af40d5a5ca56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:53.750724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:53.750724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:53.750724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:53.750724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:53.750724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:53.750724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:53.750724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:53.750724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:53.750724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:53.750724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:53.750724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:53.750724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:53.750724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:53.750724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:53.751612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:53.752472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:53.752472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:53.752472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:53.752472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:53.752472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 249/514] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:53.753141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd13d76-56c4-4210-9f93-af40d5a5ca56 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:53.815644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed33796-d1eb-46ad-8d37-c0c844ea8711 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:53.831703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:53.854557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:53.854808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2ed5b-7fd4-4c01-9f6f-089ba8742eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:53.855438 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2ed5b-7fd4-4c01-9f6f-089ba8742eeb None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:53.855739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2ed5b-7fd4-4c01-9f6f-089ba8742eeb None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:53.856180 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2ed5b-7fd4-4c01-9f6f-089ba8742eeb None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:53.856744 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2ed5b-7fd4-4c01-9f6f-089ba8742eeb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:53.858375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2ed5b-7fd4-4c01-9f6f-089ba8742eeb None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:53.864077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd13d76-56c4-4210-9f93-af40d5a5ca56 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:53.881887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 266/515] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:38:53 2022] POST /identity/v3/users => generated 247 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:38:53.882647 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2ed5b-7fd4-4c01-9f6f-089ba8742eeb None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:53.908276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bd250-5616-41f4-b2eb-1eda870ac067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:53.909109 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bd250-5616-41f4-b2eb-1eda870ac067 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:53.909387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bd250-5616-41f4-b2eb-1eda870ac067 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:53.909654 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bd250-5616-41f4-b2eb-1eda870ac067 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:53.917697 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bd250-5616-41f4-b2eb-1eda870ac067 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:53.919229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bd250-5616-41f4-b2eb-1eda870ac067 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:53.936966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-a60bd250-5616-41f4-b2eb-1eda870ac067 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:38:53.936966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:53.936966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:53.936966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:53.936966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:53.936966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:53.936966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:53.936966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:53.936966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:53.936966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:53.936966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:53.936966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:53.936966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:38:53.936966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:38:53.936966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:53.938480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:53.941204 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:38:53.941204 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:38:53.941204 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:38:53.941204 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:38:53.941204 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:38:53.941204 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:38:53.941204 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:38:53.941204 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:53.941204 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 250/516] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:53.941204 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bd250-5616-41f4-b2eb-1eda870ac067 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:54.028683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2ed5b-7fd4-4c01-9f6f-089ba8742eeb None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:54.068810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bd250-5616-41f4-b2eb-1eda870ac067 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:54.074020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b48fc-f891-455b-96d9-32b98393baf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:54.074020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b48fc-f891-455b-96d9-32b98393baf8 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:54.074020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b48fc-f891-455b-96d9-32b98393baf8 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:54.074020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b48fc-f891-455b-96d9-32b98393baf8 None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:54.074455 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b48fc-f891-455b-96d9-32b98393baf8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:54.074695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b48fc-f891-455b-96d9-32b98393baf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:54.077175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b48fc-f891-455b-96d9-32b98393baf8 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:54.119415 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05ac99-cc99-4755-93de-f0d13a652b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:54.120148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05ac99-cc99-4755-93de-f0d13a652b4c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:54.120436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05ac99-cc99-4755-93de-f0d13a652b4c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:54.120710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05ac99-cc99-4755-93de-f0d13a652b4c None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:54.121569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05ac99-cc99-4755-93de-f0d13a652b4c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:54.121877 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05ac99-cc99-4755-93de-f0d13a652b4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:54.123023 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 267/517] 10.222.0.60 () {56 vars in 1137 bytes} [Thu Jul 21 20:38:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:54.123344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05ac99-cc99-4755-93de-f0d13a652b4c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:54.124503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b48fc-f891-455b-96d9-32b98393baf8 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:54.142743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 251/518] 10.222.0.60 () {56 vars in 1137 bytes} [Thu Jul 21 20:38:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:54.143519 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05ac99-cc99-4755-93de-f0d13a652b4c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:54.158648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:54.168049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:54.253564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b48fc-f891-455b-96d9-32b98393baf8 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:54.284420 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05ac99-cc99-4755-93de-f0d13a652b4c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:54.298378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426a0a4-264e-4970-86a1-36605ab4841c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:54.299152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426a0a4-264e-4970-86a1-36605ab4841c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:54.299503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426a0a4-264e-4970-86a1-36605ab4841c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:54.299887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426a0a4-264e-4970-86a1-36605ab4841c None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:54.300702 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426a0a4-264e-4970-86a1-36605ab4841c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:54.301076 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426a0a4-264e-4970-86a1-36605ab4841c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:54.302516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426a0a4-264e-4970-86a1-36605ab4841c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:54.321697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 268/519] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:54.321697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426a0a4-264e-4970-86a1-36605ab4841c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:54.325845 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697df5a-19dc-4ff6-ba25-b846c27e56aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:54.327325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697df5a-19dc-4ff6-ba25-b846c27e56aa None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:54.327616 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697df5a-19dc-4ff6-ba25-b846c27e56aa None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:54.327924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697df5a-19dc-4ff6-ba25-b846c27e56aa None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:54.328727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697df5a-19dc-4ff6-ba25-b846c27e56aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:54.329040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697df5a-19dc-4ff6-ba25-b846c27e56aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:54.330479 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697df5a-19dc-4ff6-ba25-b846c27e56aa None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:54.355761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 252/520] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:38:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:54.355761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697df5a-19dc-4ff6-ba25-b846c27e56aa None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:54.436028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:54.440128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:54.456752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426a0a4-264e-4970-86a1-36605ab4841c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:54.472334 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697df5a-19dc-4ff6-ba25-b846c27e56aa None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:54.501725 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65449b8c-6a8d-4ddf-9fc5-28d3e6ad9f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:54.502475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65449b8c-6a8d-4ddf-9fc5-28d3e6ad9f8e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:54.502815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65449b8c-6a8d-4ddf-9fc5-28d3e6ad9f8e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:54.503096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65449b8c-6a8d-4ddf-9fc5-28d3e6ad9f8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:54.510759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a12f5f-1fdf-485f-bf18-fc14814fc933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:54.511412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a12f5f-1fdf-485f-bf18-fc14814fc933 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:54.511692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a12f5f-1fdf-485f-bf18-fc14814fc933 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:54.511993 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a12f5f-1fdf-485f-bf18-fc14814fc933 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:54.584917 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65449b8c-6a8d-4ddf-9fc5-28d3e6ad9f8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:54.586520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65449b8c-6a8d-4ddf-9fc5-28d3e6ad9f8e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:54.588814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a12f5f-1fdf-485f-bf18-fc14814fc933 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:54.590307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a12f5f-1fdf-485f-bf18-fc14814fc933 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:54.678523 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:54.682023 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:54.696280 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 269/521] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:54.700685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65449b8c-6a8d-4ddf-9fc5-28d3e6ad9f8e None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:54.733213 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 253/522] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:54.734168 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a12f5f-1fdf-485f-bf18-fc14814fc933 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:54.820378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65449b8c-6a8d-4ddf-9fc5-28d3e6ad9f8e None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:54.858143 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52337073-b90d-400f-af5b-4eee39fe90ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:54.858962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52337073-b90d-400f-af5b-4eee39fe90ef None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:54.859228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52337073-b90d-400f-af5b-4eee39fe90ef None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:54.859504 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52337073-b90d-400f-af5b-4eee39fe90ef None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:54.864956 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a12f5f-1fdf-485f-bf18-fc14814fc933 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:54.867621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52337073-b90d-400f-af5b-4eee39fe90ef None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:54.869123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52337073-b90d-400f-af5b-4eee39fe90ef None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-52337073-b90d-400f-af5b-4eee39fe90ef None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:38:54.887058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:54.888714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:54.890206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:54.890206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:54.890206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:38:54.890206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:38:54.890206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:38:54.890206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:38:54.890206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:38:54.890206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:38:54.890206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 20:38:54.890206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:54.891875 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 270/523] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:38:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:54.893573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52337073-b90d-400f-af5b-4eee39fe90ef None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:54.914970 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8fc0d-1fa8-42d2-9880-bde841408c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:54.915633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8fc0d-1fa8-42d2-9880-bde841408c6a None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:54.915971 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8fc0d-1fa8-42d2-9880-bde841408c6a None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:54.916258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8fc0d-1fa8-42d2-9880-bde841408c6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:54.917089 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8fc0d-1fa8-42d2-9880-bde841408c6a None admin] RBAC: Adding query filter params (role.id=05f3681d0a644607934e281f75ecd675, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=77812b356c974d2b80102217956169db) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:54.917400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8fc0d-1fa8-42d2-9880-bde841408c6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:54.918843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8fc0d-1fa8-42d2-9880-bde841408c6a None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:54.935843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 254/524] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:54 2022] GET /identity/v3/role_assignments?user.id=77812b356c974d2b80102217956169db&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=05f3681d0a644607934e281f75ecd675 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:54.935843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8fc0d-1fa8-42d2-9880-bde841408c6a None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:54.978204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:54.986207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:55.050308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52337073-b90d-400f-af5b-4eee39fe90ef None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:55.090839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8fc0d-1fa8-42d2-9880-bde841408c6a None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:55.100862 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ad287-f1a6-4665-b4d6-87aa5a325b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:55.102202 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ad287-f1a6-4665-b4d6-87aa5a325b58 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:55.102694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ad287-f1a6-4665-b4d6-87aa5a325b58 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:55.103171 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ad287-f1a6-4665-b4d6-87aa5a325b58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:55.116269 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ad287-f1a6-4665-b4d6-87aa5a325b58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:55.118237 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ad287-f1a6-4665-b4d6-87aa5a325b58 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:55.129411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fde321-86f1-46fc-a3cc-6de6a9c9cb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:55.130209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fde321-86f1-46fc-a3cc-6de6a9c9cb72 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:55.130490 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fde321-86f1-46fc-a3cc-6de6a9c9cb72 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:55.130838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fde321-86f1-46fc-a3cc-6de6a9c9cb72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:55.139670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-e17ad287-f1a6-4665-b4d6-87aa5a325b58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:55.139670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:55.139670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:55.139670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:55.139670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:55.139670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:55.139670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:55.139670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:55.139670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:55.139670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:55.139670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:55.139670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:55.139670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:55.139670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:55.139670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:55.140610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:55.141534 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:55.141534 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:55.141534 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:55.141534 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:55.142375 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fde321-86f1-46fc-a3cc-6de6a9c9cb72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:55.143880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fde321-86f1-46fc-a3cc-6de6a9c9cb72 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:55.145580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 271/525] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:55.146559 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ad287-f1a6-4665-b4d6-87aa5a325b58 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:55.167551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-74fde321-86f1-46fc-a3cc-6de6a9c9cb72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:55.167551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:55.167551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:55.167551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:55.167551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:55.167551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:55.167551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:55.167551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:55.167551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:55.167551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:55.167551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:55.167551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:55.167551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:55.167551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:55.167551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:55.168657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:55.169471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:55.169471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:55.169471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:55.169471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:55.169471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 255/526] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:55.175756 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fde321-86f1-46fc-a3cc-6de6a9c9cb72 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:55.277436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:55.281182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ad287-f1a6-4665-b4d6-87aa5a325b58 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:55.286151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:55.307088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fde321-86f1-46fc-a3cc-6de6a9c9cb72 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:55.330958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae469d88-aa9d-452b-bd06-97171892ed1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:55.331637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae469d88-aa9d-452b-bd06-97171892ed1c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:55.331967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae469d88-aa9d-452b-bd06-97171892ed1c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:55.332245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae469d88-aa9d-452b-bd06-97171892ed1c None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:55.333110 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae469d88-aa9d-452b-bd06-97171892ed1c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:55.333961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae469d88-aa9d-452b-bd06-97171892ed1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:55.335847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae469d88-aa9d-452b-bd06-97171892ed1c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:55.349181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26df30e-b976-4ffa-8048-a9aac1bfe12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:55.350780 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26df30e-b976-4ffa-8048-a9aac1bfe12d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:55.351339 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26df30e-b976-4ffa-8048-a9aac1bfe12d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:55.351846 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26df30e-b976-4ffa-8048-a9aac1bfe12d None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:55.352788 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26df30e-b976-4ffa-8048-a9aac1bfe12d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:55.353298 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26df30e-b976-4ffa-8048-a9aac1bfe12d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:55.355480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26df30e-b976-4ffa-8048-a9aac1bfe12d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:55.359491 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 272/527] 10.222.0.60 () {56 vars in 1139 bytes} [Thu Jul 21 20:38:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:55.360396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae469d88-aa9d-452b-bd06-97171892ed1c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:55.370323 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 256/528] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:55.371740 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26df30e-b976-4ffa-8048-a9aac1bfe12d None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:55.486639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae469d88-aa9d-452b-bd06-97171892ed1c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:55.535706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26df30e-b976-4ffa-8048-a9aac1bfe12d None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:55.543147 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bc5c0-814a-41b4-8af1-e1248f8a29f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:55.544191 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bc5c0-814a-41b4-8af1-e1248f8a29f2 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:55.544708 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bc5c0-814a-41b4-8af1-e1248f8a29f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:55.545186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bc5c0-814a-41b4-8af1-e1248f8a29f2 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:55.546119 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bc5c0-814a-41b4-8af1-e1248f8a29f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:55.546630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bc5c0-814a-41b4-8af1-e1248f8a29f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:55.548294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bc5c0-814a-41b4-8af1-e1248f8a29f2 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:55.562024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 273/529] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:55.563201 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bc5c0-814a-41b4-8af1-e1248f8a29f2 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:55.591233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9912f6-f1d1-4b0e-b539-cf8c45ac4770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:55.591924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9912f6-f1d1-4b0e-b539-cf8c45ac4770 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:55.592214 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9912f6-f1d1-4b0e-b539-cf8c45ac4770 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:55.592518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9912f6-f1d1-4b0e-b539-cf8c45ac4770 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:55.599226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:55.618255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:55.695609 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bc5c0-814a-41b4-8af1-e1248f8a29f2 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:55.698451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9912f6-f1d1-4b0e-b539-cf8c45ac4770 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:55.700957 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9912f6-f1d1-4b0e-b539-cf8c45ac4770 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:55.741143 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e90b0-145b-44ec-b52c-c909654974ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:55.742388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e90b0-145b-44ec-b52c-c909654974ba None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:55.742875 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e90b0-145b-44ec-b52c-c909654974ba None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:55.743393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e90b0-145b-44ec-b52c-c909654974ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:55.751522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e90b0-145b-44ec-b52c-c909654974ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:55.753359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e90b0-145b-44ec-b52c-c909654974ba None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:55.769656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-467e90b0-145b-44ec-b52c-c909654974ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:38:55.769656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:55.769656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:55.769656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:55.769656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:55.769656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:55.769656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:55.769656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:55.769656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:55.769656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:55.769656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:55.769656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:55.769656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:38:55.769656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:38:55.769656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:55.770546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:55.771946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:38:55.771946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:38:55.771946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:38:55.771946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:38:55.771946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:38:55.771946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:38:55.771946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:38:55.771946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:55.774599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 274/530] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:55.775754 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e90b0-145b-44ec-b52c-c909654974ba None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:55.847942 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 257/531] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:38:55.849555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9912f6-f1d1-4b0e-b539-cf8c45ac4770 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:55.884280 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:55.913808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:55.918127 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e90b0-145b-44ec-b52c-c909654974ba None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:55.955884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c69fb-052d-4125-8948-c516b43b979e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:55.956701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c69fb-052d-4125-8948-c516b43b979e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:55.956987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c69fb-052d-4125-8948-c516b43b979e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:55.957266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c69fb-052d-4125-8948-c516b43b979e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:55.965222 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c69fb-052d-4125-8948-c516b43b979e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:55.966715 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c69fb-052d-4125-8948-c516b43b979e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:55.983210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-b76c69fb-052d-4125-8948-c516b43b979e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:38:55.983210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:55.983210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:55.983210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:55.983210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:55.983210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:55.983210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:55.983210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:55.983210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:55.983210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:55.983210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:55.983210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:55.983210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:38:55.983210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:38:55.983210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:55.984567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:55.985393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:38:55.985393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:38:55.985393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:38:55.985393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:38:55.985393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:38:55.985393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:38:55.985393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:38:55.985393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:55.985393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 275/532] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:55.985854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c69fb-052d-4125-8948-c516b43b979e None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:55.993214 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9912f6-f1d1-4b0e-b539-cf8c45ac4770 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:56.034186 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59375559-a8c5-4748-8a1b-7edffb69720f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:56.035223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59375559-a8c5-4748-8a1b-7edffb69720f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:56.035705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59375559-a8c5-4748-8a1b-7edffb69720f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:56.036275 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59375559-a8c5-4748-8a1b-7edffb69720f None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:56.037904 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59375559-a8c5-4748-8a1b-7edffb69720f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:56.038416 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59375559-a8c5-4748-8a1b-7edffb69720f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:56.040122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59375559-a8c5-4748-8a1b-7edffb69720f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:56.054005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 258/533] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:38:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:56.055267 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59375559-a8c5-4748-8a1b-7edffb69720f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:56.106339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c69fb-052d-4125-8948-c516b43b979e None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:56.147082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775258c4-3aab-4b6f-a34a-2e677e07bcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:56.147927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775258c4-3aab-4b6f-a34a-2e677e07bcd9 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:56.148203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775258c4-3aab-4b6f-a34a-2e677e07bcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:56.148576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775258c4-3aab-4b6f-a34a-2e677e07bcd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:56.156142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:56.161561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775258c4-3aab-4b6f-a34a-2e677e07bcd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:56.163223 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775258c4-3aab-4b6f-a34a-2e677e07bcd9 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:56.174684 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59375559-a8c5-4748-8a1b-7edffb69720f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:56.182680 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:56.189041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-775258c4-3aab-4b6f-a34a-2e677e07bcd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:56.189041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:56.189041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:56.189041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:56.189041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:56.189041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:56.189041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:56.189041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:56.189041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:56.189041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:56.189041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:56.189041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:56.189041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:56.189041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:56.189041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:56.189890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:56.190716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:56.190716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:56.190716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:56.190716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:56.190716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 276/534] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:56.191912 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa43cc-ec9f-42c2-8056-1d7b2117e7e6 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:56.192199 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa43cc-ec9f-42c2-8056-1d7b2117e7e6 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:56.192475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa43cc-ec9f-42c2-8056-1d7b2117e7e6 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:56.194426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 277/535] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:56.194984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa43cc-ec9f-42c2-8056-1d7b2117e7e6 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:56.224076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86e56c-c5bf-4ad5-b218-972c22dcb02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:56.224834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86e56c-c5bf-4ad5-b218-972c22dcb02c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:56.225350 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86e56c-c5bf-4ad5-b218-972c22dcb02c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:56.225690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86e56c-c5bf-4ad5-b218-972c22dcb02c None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:56.226495 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86e56c-c5bf-4ad5-b218-972c22dcb02c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:56.226808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86e56c-c5bf-4ad5-b218-972c22dcb02c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:56.228303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86e56c-c5bf-4ad5-b218-972c22dcb02c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:56.242523 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 259/536] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:38:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:56.243197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86e56c-c5bf-4ad5-b218-972c22dcb02c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:56.333018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa43cc-ec9f-42c2-8056-1d7b2117e7e6 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:56.369657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86e56c-c5bf-4ad5-b218-972c22dcb02c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:56.375538 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fb0cf-0414-4236-9fd9-e3608843ee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:56.376412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fb0cf-0414-4236-9fd9-e3608843ee1a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:56.376960 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fb0cf-0414-4236-9fd9-e3608843ee1a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:56.377244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fb0cf-0414-4236-9fd9-e3608843ee1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:56.378086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fb0cf-0414-4236-9fd9-e3608843ee1a None admin] RBAC: Adding query filter params (role.id=05f3681d0a644607934e281f75ecd675, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=a8eef7746fc4417e8d4de0b052f3eaa0) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:56.378399 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fb0cf-0414-4236-9fd9-e3608843ee1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:56.379889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fb0cf-0414-4236-9fd9-e3608843ee1a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:56.387034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 278/537] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:56 2022] GET /identity/v3/role_assignments?user.id=a8eef7746fc4417e8d4de0b052f3eaa0&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=05f3681d0a644607934e281f75ecd675 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:56.387034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab07031-6811-4b7b-a1c0-773b6f2d9689 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:56.387034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab07031-6811-4b7b-a1c0-773b6f2d9689 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:56.387034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab07031-6811-4b7b-a1c0-773b6f2d9689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:56.414487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-fab07031-6811-4b7b-a1c0-773b6f2d9689 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:56.415933 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fab07031-6811-4b7b-a1c0-773b6f2d9689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:56.433652 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a98d7a-b813-4734-bc50-c1ce3d2eac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:56.434336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a98d7a-b813-4734-bc50-c1ce3d2eac9d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:56.434617 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a98d7a-b813-4734-bc50-c1ce3d2eac9d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:56.434890 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a98d7a-b813-4734-bc50-c1ce3d2eac9d None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:56.435762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a98d7a-b813-4734-bc50-c1ce3d2eac9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:56.436084 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a98d7a-b813-4734-bc50-c1ce3d2eac9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:56.437497 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a98d7a-b813-4734-bc50-c1ce3d2eac9d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:56.460158 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 260/538] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:56.460158 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a98d7a-b813-4734-bc50-c1ce3d2eac9d None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:56.494025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:56.496039 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:56.519465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 279/539] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:56.528963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab07031-6811-4b7b-a1c0-773b6f2d9689 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:56.579603 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a98d7a-b813-4734-bc50-c1ce3d2eac9d None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:56.618520 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aed014-6474-4d3f-be6b-b0b69634353b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:56.619470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aed014-6474-4d3f-be6b-b0b69634353b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:56.620004 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aed014-6474-4d3f-be6b-b0b69634353b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:56.620220 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aed014-6474-4d3f-be6b-b0b69634353b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:56.621103 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aed014-6474-4d3f-be6b-b0b69634353b None admin] RBAC: Adding query filter params (role.id=05f3681d0a644607934e281f75ecd675, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=1d596de177e24ca5a254bda6644a07da) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:56.621393 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aed014-6474-4d3f-be6b-b0b69634353b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:56.622841 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aed014-6474-4d3f-be6b-b0b69634353b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:56.629251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 261/540] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:56 2022] GET /identity/v3/role_assignments?user.id=1d596de177e24ca5a254bda6644a07da&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=05f3681d0a644607934e281f75ecd675 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:56.630098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aed014-6474-4d3f-be6b-b0b69634353b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:56.695805 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab07031-6811-4b7b-a1c0-773b6f2d9689 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:56.781871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc07c43-89ab-4447-9e69-f77544a199d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:56.782455 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:56.782883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc07c43-89ab-4447-9e69-f77544a199d5 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:56.782983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc07c43-89ab-4447-9e69-f77544a199d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:56.783255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc07c43-89ab-4447-9e69-f77544a199d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:56.799903 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc07c43-89ab-4447-9e69-f77544a199d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:56.801792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc07c43-89ab-4447-9e69-f77544a199d5 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:56.808513 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aed014-6474-4d3f-be6b-b0b69634353b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:56.829011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-ebc07c43-89ab-4447-9e69-f77544a199d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:38:56.829011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:56.829011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:56.829011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:56.829011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:56.829011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:56.829011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:56.829011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:56.829011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:56.829011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:56.829011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:56.829011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:56.829011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:38:56.829011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:38:56.829011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:56.829916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:56.831044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:38:56.831044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:38:56.831044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:38:56.831044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:38:56.831044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:38:56.831044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:38:56.831044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:38:56.831044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:56.831044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 280/541] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:56.836205 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc07c43-89ab-4447-9e69-f77544a199d5 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:56.874948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115a1cc-6041-4f76-a514-259d3690d462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:56.876261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115a1cc-6041-4f76-a514-259d3690d462 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:56.876694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115a1cc-6041-4f76-a514-259d3690d462 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:56.877133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115a1cc-6041-4f76-a514-259d3690d462 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:56.889784 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115a1cc-6041-4f76-a514-259d3690d462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:56.893584 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-f115a1cc-6041-4f76-a514-259d3690d462 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:56.901317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:56.902352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:56.903311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:56.903311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:56.903311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:56.903311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:56.903311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:56.903311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:56.903311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:56.903311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:56.903311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:56.903311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:56.903311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:38:56.903311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:56.910390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 262/542] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:38:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:56.911918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115a1cc-6041-4f76-a514-259d3690d462 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:57.008935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc07c43-89ab-4447-9e69-f77544a199d5 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:57.040600 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115a1cc-6041-4f76-a514-259d3690d462 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:57.081341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1692b-178b-465c-9845-95c8429c4bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:57.082008 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1692b-178b-465c-9845-95c8429c4bfb None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:57.082292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1692b-178b-465c-9845-95c8429c4bfb None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:57.082569 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1692b-178b-465c-9845-95c8429c4bfb None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:57.083369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1692b-178b-465c-9845-95c8429c4bfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:57.083682 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1692b-178b-465c-9845-95c8429c4bfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:57.085695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1692b-178b-465c-9845-95c8429c4bfb None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:57.111897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614f830-44fc-4675-ae4f-6cd4e8feac1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:57.112571 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614f830-44fc-4675-ae4f-6cd4e8feac1f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:57.112855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614f830-44fc-4675-ae4f-6cd4e8feac1f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:57.113130 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614f830-44fc-4675-ae4f-6cd4e8feac1f None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:57.113893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614f830-44fc-4675-ae4f-6cd4e8feac1f None admin] RBAC: Adding query filter params (name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:57.114198 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614f830-44fc-4675-ae4f-6cd4e8feac1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:57.115629 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614f830-44fc-4675-ae4f-6cd4e8feac1f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:57.127970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 281/543] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:38:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:57.134813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1692b-178b-465c-9845-95c8429c4bfb None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:57.137672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 263/544] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:38:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:57.147510 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614f830-44fc-4675-ae4f-6cd4e8feac1f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:57.169527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:57.283253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1692b-178b-465c-9845-95c8429c4bfb None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:57.287837 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:57.293766 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614f830-44fc-4675-ae4f-6cd4e8feac1f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:57.329251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c13f89-3abb-40fb-a745-ae68e0e6fa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:57.329920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c13f89-3abb-40fb-a745-ae68e0e6fa8c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:57.330488 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c13f89-3abb-40fb-a745-ae68e0e6fa8c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:57.330488 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c13f89-3abb-40fb-a745-ae68e0e6fa8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:57.331360 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c13f89-3abb-40fb-a745-ae68e0e6fa8c None admin] RBAC: Adding query filter params (role.id=05f3681d0a644607934e281f75ecd675, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=33aff48933844401b86eb808937e35c4) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:57.331654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c13f89-3abb-40fb-a745-ae68e0e6fa8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:57.333449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c13f89-3abb-40fb-a745-ae68e0e6fa8c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:57.339406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29091f57-f49b-40ae-9755-8c73ac58b0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:57.340515 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29091f57-f49b-40ae-9755-8c73ac58b0f7 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:57.340972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29091f57-f49b-40ae-9755-8c73ac58b0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:57.341443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29091f57-f49b-40ae-9755-8c73ac58b0f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:57.343804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 282/545] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:38:57 2022] GET /identity/v3/role_assignments?user.id=33aff48933844401b86eb808937e35c4&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=05f3681d0a644607934e281f75ecd675 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:57.345519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728efb4-dfa0-46f5-95b1-de8125b11e3d None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:57.346000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728efb4-dfa0-46f5-95b1-de8125b11e3d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:57.346450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728efb4-dfa0-46f5-95b1-de8125b11e3d None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:57.356184 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 283/546] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:57 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:57.362218 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07613998-b1e2-4de7-be94-e3fc6045eda9 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:57.362795 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07613998-b1e2-4de7-be94-e3fc6045eda9 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:57.363248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07613998-b1e2-4de7-be94-e3fc6045eda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:57.368879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29091f57-f49b-40ae-9755-8c73ac58b0f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:57.373819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29091f57-f49b-40ae-9755-8c73ac58b0f7 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:57.398810 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-07613998-b1e2-4de7-be94-e3fc6045eda9 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:57.405383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-07613998-b1e2-4de7-be94-e3fc6045eda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:57.407662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-29091f57-f49b-40ae-9755-8c73ac58b0f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:57.407662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:57.407662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:57.407662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:57.407662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:57.407662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:57.407662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:57.407662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:57.407662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:57.407662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:57.407662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:57.407662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:57.407662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:57.407662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:57.407662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:57.408682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:57.409621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:57.409621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:57.409621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:57.409621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:57.417217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 264/547] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:57.418537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29091f57-f49b-40ae-9755-8c73ac58b0f7 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:57.492045 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:57.538540 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29091f57-f49b-40ae-9755-8c73ac58b0f7 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:57.582656 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b5164-9c61-4eb2-b0e8-33a42e93f0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:57.583355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b5164-9c61-4eb2-b0e8-33a42e93f0b7 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:57.583677 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b5164-9c61-4eb2-b0e8-33a42e93f0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:57.584575 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b5164-9c61-4eb2-b0e8-33a42e93f0b7 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:57.585254 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b5164-9c61-4eb2-b0e8-33a42e93f0b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:57.585610 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b5164-9c61-4eb2-b0e8-33a42e93f0b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:57.586384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 284/548] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:57.587163 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b5164-9c61-4eb2-b0e8-33a42e93f0b7 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:57.610180 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07613998-b1e2-4de7-be94-e3fc6045eda9 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:57.612460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 265/549] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:57.618261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b5164-9c61-4eb2-b0e8-33a42e93f0b7 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:57.779761 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:57.786509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07613998-b1e2-4de7-be94-e3fc6045eda9 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:57.809767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b5164-9c61-4eb2-b0e8-33a42e93f0b7 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:57.828615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d67990-f74d-4bc7-aa87-75aaf361fdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:57.829821 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d67990-f74d-4bc7-aa87-75aaf361fdad None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:57.830301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d67990-f74d-4bc7-aa87-75aaf361fdad None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:57.830771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d67990-f74d-4bc7-aa87-75aaf361fdad None admin] PATH_INFO: `/v3/roles/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:57.842971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d67990-f74d-4bc7-aa87-75aaf361fdad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-80d67990-f74d-4bc7-aa87-75aaf361fdad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:57.852075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:57.853500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:57.854716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:57.854716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:57.854716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:57.854716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:57.854716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:57.854716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:57.854716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:57.854716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:57.854716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:57.854716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:57.854716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:38:57.854716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:57.858014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 285/550] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:38:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:38:57.865580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d67990-f74d-4bc7-aa87-75aaf361fdad None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:57.885650 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:57.903438 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626ae2f-5b2c-4553-8687-2cb2a720c5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:57.904275 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626ae2f-5b2c-4553-8687-2cb2a720c5a6 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:57.904562 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626ae2f-5b2c-4553-8687-2cb2a720c5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:57.904843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626ae2f-5b2c-4553-8687-2cb2a720c5a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:57.925973 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626ae2f-5b2c-4553-8687-2cb2a720c5a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:57.928028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626ae2f-5b2c-4553-8687-2cb2a720c5a6 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-c626ae2f-5b2c-4553-8687-2cb2a720c5a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:38:57.961988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:57.962948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:57.964025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:57.964025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:57.964025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:38:57.964025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:38:57.964025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:38:57.964025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:38:57.964025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:38:57.964025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:38:57.964025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:38:57.964025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:57.964025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 266/551] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:38:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:57.970312 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626ae2f-5b2c-4553-8687-2cb2a720c5a6 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:58.101973 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626ae2f-5b2c-4553-8687-2cb2a720c5a6 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:58.125949 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d67990-f74d-4bc7-aa87-75aaf361fdad None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:58.143308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7db08-63b3-466a-82fb-4c3fa8b3f10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:58.144118 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7db08-63b3-466a-82fb-4c3fa8b3f10c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:58.144467 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7db08-63b3-466a-82fb-4c3fa8b3f10c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:58.144760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7db08-63b3-466a-82fb-4c3fa8b3f10c None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:58.145830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7db08-63b3-466a-82fb-4c3fa8b3f10c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:58.146807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7db08-63b3-466a-82fb-4c3fa8b3f10c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:58.148980 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7db08-63b3-466a-82fb-4c3fa8b3f10c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:58.175598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 267/552] 10.222.0.60 () {56 vars in 1135 bytes} [Thu Jul 21 20:38:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:58.197711 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7db08-63b3-466a-82fb-4c3fa8b3f10c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:58.217451 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a2f91-2ab7-4639-8145-176dc6e238cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:58.218147 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a2f91-2ab7-4639-8145-176dc6e238cd None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:58.218494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a2f91-2ab7-4639-8145-176dc6e238cd None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:58.218774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a2f91-2ab7-4639-8145-176dc6e238cd None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:58.219605 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a2f91-2ab7-4639-8145-176dc6e238cd None admin] RBAC: Adding query filter params (name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:58.219943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a2f91-2ab7-4639-8145-176dc6e238cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:58.221404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a2f91-2ab7-4639-8145-176dc6e238cd None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:58.241775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 286/553] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:38:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:58.247523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a2f91-2ab7-4639-8145-176dc6e238cd None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:58.249781 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:58.295469 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:58.315868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7db08-63b3-466a-82fb-4c3fa8b3f10c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:58.354101 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d46bb-1b22-412f-8e01-64519edae03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:58.355370 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d46bb-1b22-412f-8e01-64519edae03e None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:58.355878 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d46bb-1b22-412f-8e01-64519edae03e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:58.356368 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d46bb-1b22-412f-8e01-64519edae03e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:58.414031 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a2f91-2ab7-4639-8145-176dc6e238cd None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:58.451456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22edb6-e4cc-4576-bb29-75ef53bd6c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:58.452281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22edb6-e4cc-4576-bb29-75ef53bd6c10 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:58.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22edb6-e4cc-4576-bb29-75ef53bd6c10 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:58.452834 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22edb6-e4cc-4576-bb29-75ef53bd6c10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:58.464633 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22edb6-e4cc-4576-bb29-75ef53bd6c10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:58.466219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22edb6-e4cc-4576-bb29-75ef53bd6c10 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:58.485426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-fc22edb6-e4cc-4576-bb29-75ef53bd6c10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:58.485426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:58.485426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:58.485426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:58.485426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:58.485426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:58.485426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:58.485426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:58.485426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:58.485426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:58.485426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:58.485426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:58.485426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:58.485426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:58.485426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:58.486384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:58.487413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:58.487413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:58.487413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:58.487413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:58.487413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 287/554] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:38:58.493575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22edb6-e4cc-4576-bb29-75ef53bd6c10 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:58.518076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d46bb-1b22-412f-8e01-64519edae03e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:58.519560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d46bb-1b22-412f-8e01-64519edae03e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:58.606420 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:58.664909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:58.723200 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22edb6-e4cc-4576-bb29-75ef53bd6c10 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:58.727226 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 268/555] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:38:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 20:38:58.732659 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d46bb-1b22-412f-8e01-64519edae03e None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:58.777241 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937f173-87f9-41c9-aac6-f8826c55c6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:58.777911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937f173-87f9-41c9-aac6-f8826c55c6ce None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:58.778193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937f173-87f9-41c9-aac6-f8826c55c6ce None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:58.778462 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937f173-87f9-41c9-aac6-f8826c55c6ce None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:58.779192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937f173-87f9-41c9-aac6-f8826c55c6ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:58.779493 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937f173-87f9-41c9-aac6-f8826c55c6ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:58.780915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937f173-87f9-41c9-aac6-f8826c55c6ce None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:58.795469 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 288/556] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:38:58.807962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937f173-87f9-41c9-aac6-f8826c55c6ce None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:58.919963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d46bb-1b22-412f-8e01-64519edae03e None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:58.954458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937f173-87f9-41c9-aac6-f8826c55c6ce None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:58.974222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eda1cd-43ee-4819-bf55-802b30225eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:58.975652 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eda1cd-43ee-4819-bf55-802b30225eaf None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:58.976331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eda1cd-43ee-4819-bf55-802b30225eaf None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:58.976673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eda1cd-43ee-4819-bf55-802b30225eaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:58.991502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eda1cd-43ee-4819-bf55-802b30225eaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:58.993408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eda1cd-43ee-4819-bf55-802b30225eaf None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:58.995473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:58.997571 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:59.005221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79002306-1e12-4994-8ab6-75d7b8c169df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:59.007018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79002306-1e12-4994-8ab6-75d7b8c169df None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:59.007385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79002306-1e12-4994-8ab6-75d7b8c169df None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:59.009921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79002306-1e12-4994-8ab6-75d7b8c169df None admin] PATH_INFO: `/v3/users/nova` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:59.020453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79002306-1e12-4994-8ab6-75d7b8c169df None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:59.022938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79002306-1e12-4994-8ab6-75d7b8c169df None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:59.024005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-75eda1cd-43ee-4819-bf55-802b30225eaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:59.024005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:59.024005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:59.024005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:59.024005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:59.024005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:59.024005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:59.024005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:59.024005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:59.024005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:59.024005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:59.024005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:59.024005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:59.024005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:59.024005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:59.025566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:59.026696 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:59.026696 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:59.026696 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:59.026696 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:59.029834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 269/557] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:59.031477 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803a69-6ee9-421e-a847-bc4a23dd9ade None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:59.032015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803a69-6ee9-421e-a847-bc4a23dd9ade None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:59.032421 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803a69-6ee9-421e-a847-bc4a23dd9ade None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:59.038828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 270/558] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:59.040044 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e803a69-6ee9-421e-a847-bc4a23dd9ade None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-79002306-1e12-4994-8ab6-75d7b8c169df None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:38:59.047457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:59.048704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:59.049932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:59.049932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:59.049932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:38:59.049932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:38:59.049932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:38:59.049932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:38:59.049932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:38:59.049932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:38:59.049932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:38:59.049932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:59.057203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 289/559] 10.222.0.60 () {56 vars in 1093 bytes} [Thu Jul 21 20:38:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:59.058745 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff16b3-3ae8-4613-8b50-fb06ba47d6ca None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:59.059139 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff16b3-3ae8-4613-8b50-fb06ba47d6ca None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:59.059546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff16b3-3ae8-4613-8b50-fb06ba47d6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:59.078737 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-74ff16b3-3ae8-4613-8b50-fb06ba47d6ca None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:59.080398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-74ff16b3-3ae8-4613-8b50-fb06ba47d6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:59.169064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e803a69-6ee9-421e-a847-bc4a23dd9ade None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:59.181813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 290/560] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:59.183119 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff16b3-3ae8-4613-8b50-fb06ba47d6ca None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:59.209147 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee94191-6427-4a99-8a69-f1f814d88643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:59.210148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee94191-6427-4a99-8a69-f1f814d88643 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:59.210607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee94191-6427-4a99-8a69-f1f814d88643 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:59.211054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee94191-6427-4a99-8a69-f1f814d88643 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:59.212053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee94191-6427-4a99-8a69-f1f814d88643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:59.212558 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee94191-6427-4a99-8a69-f1f814d88643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:59.214206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee94191-6427-4a99-8a69-f1f814d88643 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:59.228158 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 271/561] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:59.229641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee94191-6427-4a99-8a69-f1f814d88643 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:59.305701 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:59.327409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff16b3-3ae8-4613-8b50-fb06ba47d6ca None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:59.357692 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:59.372817 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee94191-6427-4a99-8a69-f1f814d88643 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:59.375453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04efb4c-e6f4-40ba-bf06-cd00d2444e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:59.376413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04efb4c-e6f4-40ba-bf06-cd00d2444e18 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:59.376935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04efb4c-e6f4-40ba-bf06-cd00d2444e18 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:59.377416 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04efb4c-e6f4-40ba-bf06-cd00d2444e18 None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:59.378523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04efb4c-e6f4-40ba-bf06-cd00d2444e18 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:59.379043 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04efb4c-e6f4-40ba-bf06-cd00d2444e18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:59.380855 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04efb4c-e6f4-40ba-bf06-cd00d2444e18 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:59.408279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 291/562] 10.222.0.60 () {56 vars in 1133 bytes} [Thu Jul 21 20:38:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:59.409323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04efb4c-e6f4-40ba-bf06-cd00d2444e18 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:59.416231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e149f5c-093f-4953-97b8-ce34d31e3fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:59.418062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e149f5c-093f-4953-97b8-ce34d31e3fa9 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:59.418960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e149f5c-093f-4953-97b8-ce34d31e3fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:59.419698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e149f5c-093f-4953-97b8-ce34d31e3fa9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:59.438330 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e149f5c-093f-4953-97b8-ce34d31e3fa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-8e149f5c-093f-4953-97b8-ce34d31e3fa9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:38:59.448221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:59.449189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:59.450017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:59.450017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:59.450017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:59.450017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:59.450017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:38:59.450017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:38:59.450017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:38:59.450017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:38:59.450017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:38:59.450017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:38:59.450017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:38:59.450017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:59.450017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 272/563] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:38:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:59.450694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e149f5c-093f-4953-97b8-ce34d31e3fa9 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:59.543088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04efb4c-e6f4-40ba-bf06-cd00d2444e18 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:59.569047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e149f5c-093f-4953-97b8-ce34d31e3fa9 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:59.581772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1451fb-099a-4c0d-80e8-a869862bded7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:59.582593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1451fb-099a-4c0d-80e8-a869862bded7 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:59.582881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1451fb-099a-4c0d-80e8-a869862bded7 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:59.583155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1451fb-099a-4c0d-80e8-a869862bded7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:59.592819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:59.603376 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1451fb-099a-4c0d-80e8-a869862bded7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:59.603376 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1451fb-099a-4c0d-80e8-a869862bded7 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:59.611730 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9c055-9db7-49f8-9d07-ef2788c741ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:59.615559 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9c055-9db7-49f8-9d07-ef2788c741ac None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:59.615559 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9c055-9db7-49f8-9d07-ef2788c741ac None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:59.615559 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9c055-9db7-49f8-9d07-ef2788c741ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:59.617253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-9b1451fb-099a-4c0d-80e8-a869862bded7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:38:59.617253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:59.617253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:59.617253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:59.617253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:59.617253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:59.617253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:59.617253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:59.617253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:59.617253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:59.617253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:59.617253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:59.617253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:38:59.617253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:38:59.617253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:59.618152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:59.619094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:38:59.619094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:38:59.619094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:38:59.619094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:38:59.619094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:38:59.619094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:38:59.619094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:38:59.619094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:38:59.619094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 292/564] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:38:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:59.624514 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d95a0-b1c3-4cbb-b6bc-2d74caed3472 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:59.624803 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d95a0-b1c3-4cbb-b6bc-2d74caed3472 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:59.625084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d95a0-b1c3-4cbb-b6bc-2d74caed3472 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:59.626586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 293/565] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:59.627148 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d95a0-b1c3-4cbb-b6bc-2d74caed3472 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:59.634212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9c055-9db7-49f8-9d07-ef2788c741ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:59.636072 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9c055-9db7-49f8-9d07-ef2788c741ac None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:59.654445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-c5f9c055-9db7-49f8-9d07-ef2788c741ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:59.654445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:38:59.654445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:38:59.654445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:38:59.654445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:38:59.654445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:38:59.654445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:38:59.654445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:38:59.654445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:38:59.654445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:38:59.654445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:38:59.654445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:38:59.654445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:38:59.654445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:38:59.654445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:38:59.655608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:38:59.656565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:38:59.656565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:38:59.656565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:38:59.656565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:38:59.656565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 273/566] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:38:59.656565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9c055-9db7-49f8-9d07-ef2788c741ac None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:59.677944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:59.739114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d95a0-b1c3-4cbb-b6bc-2d74caed3472 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:59.774104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9c055-9db7-49f8-9d07-ef2788c741ac None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:59.775129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2d384-e674-4208-8ed6-2f4c95637d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:59.775826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2d384-e674-4208-8ed6-2f4c95637d47 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:59.776108 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2d384-e674-4208-8ed6-2f4c95637d47 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:59.776380 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2d384-e674-4208-8ed6-2f4c95637d47 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:59.777135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2d384-e674-4208-8ed6-2f4c95637d47 None admin] RBAC: Adding query filter params (name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:59.777437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2d384-e674-4208-8ed6-2f4c95637d47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:59.778816 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2d384-e674-4208-8ed6-2f4c95637d47 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:59.787915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 294/567] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:38:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:59.789303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e179c-ffb5-4cce-a2ea-5a5bc1d5d949 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:59.789723 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e179c-ffb5-4cce-a2ea-5a5bc1d5d949 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:59.790067 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e179c-ffb5-4cce-a2ea-5a5bc1d5d949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:59.807491 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e43e179c-ffb5-4cce-a2ea-5a5bc1d5d949 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:38:59.808734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e43e179c-ffb5-4cce-a2ea-5a5bc1d5d949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:38:59.819307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443adb8-e4dc-4ba7-96d0-451478715166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:59.820104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443adb8-e4dc-4ba7-96d0-451478715166 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:59.820387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443adb8-e4dc-4ba7-96d0-451478715166 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:59.820661 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443adb8-e4dc-4ba7-96d0-451478715166 None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:59.821461 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443adb8-e4dc-4ba7-96d0-451478715166 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:59.821765 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443adb8-e4dc-4ba7-96d0-451478715166 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:38:59.823201 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443adb8-e4dc-4ba7-96d0-451478715166 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:38:59.836865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 274/568] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:38:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:38:59.838281 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ddbcd-95ce-4ed3-b8fb-e19fa559b8f7 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:59.838573 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ddbcd-95ce-4ed3-b8fb-e19fa559b8f7 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:59.838846 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ddbcd-95ce-4ed3-b8fb-e19fa559b8f7 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:59.840131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 275/569] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:38:59.840883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ddbcd-95ce-4ed3-b8fb-e19fa559b8f7 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:59.870743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:59.887608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 295/570] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:38:59.888596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e179c-ffb5-4cce-a2ea-5a5bc1d5d949 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:38:59.923894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:38:59.958255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ddbcd-95ce-4ed3-b8fb-e19fa559b8f7 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:59.995888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f75097-68b9-4bac-934d-0c41df8d1263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:38:59.996922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f75097-68b9-4bac-934d-0c41df8d1263 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:38:59.997028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e179c-ffb5-4cce-a2ea-5a5bc1d5d949 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:38:59.997382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f75097-68b9-4bac-934d-0c41df8d1263 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:38:59.997808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f75097-68b9-4bac-934d-0c41df8d1263 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:38:59.998561 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f75097-68b9-4bac-934d-0c41df8d1263 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:38:59.998875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f75097-68b9-4bac-934d-0c41df8d1263 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:00.000300 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f75097-68b9-4bac-934d-0c41df8d1263 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:00.013064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 276/571] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:00.014361 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07030e27-7652-4af8-8a91-674f048cdea2 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:00.014652 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07030e27-7652-4af8-8a91-674f048cdea2 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:00.014939 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07030e27-7652-4af8-8a91-674f048cdea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:00.031869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-07030e27-7652-4af8-8a91-674f048cdea2 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:00.033086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-07030e27-7652-4af8-8a91-674f048cdea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:00.034325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d2bf-e4ce-42fe-858c-47d5df7a879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:00.035169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d2bf-e4ce-42fe-858c-47d5df7a879c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:00.035452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d2bf-e4ce-42fe-858c-47d5df7a879c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:00.035751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9d2bf-e4ce-42fe-858c-47d5df7a879c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:00.046666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9d2bf-e4ce-42fe-858c-47d5df7a879c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:00.048152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9d2bf-e4ce-42fe-858c-47d5df7a879c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:00.067307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-a2d9d2bf-e4ce-42fe-858c-47d5df7a879c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:00.067307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:00.067307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:00.067307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:00.067307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:00.067307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:00.067307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:00.067307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:00.067307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:00.067307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:00.067307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:00.067307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:00.067307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:00.067307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:00.067307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:00.069394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:00.070588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:00.070588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:00.070588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:00.070588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:00.070588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 296/572] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:00.070964 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d2bf-e4ce-42fe-858c-47d5df7a879c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:00.109699 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 277/573] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:39:00.110553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07030e27-7652-4af8-8a91-674f048cdea2 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:00.131747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:00.165188 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:00.181931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9d2bf-e4ce-42fe-858c-47d5df7a879c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:00.218477 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23645d17-8f52-43c3-8c64-f1a871603c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:00.219133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23645d17-8f52-43c3-8c64-f1a871603c3f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:00.219411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23645d17-8f52-43c3-8c64-f1a871603c3f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:00.219689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23645d17-8f52-43c3-8c64-f1a871603c3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:00.220553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23645d17-8f52-43c3-8c64-f1a871603c3f None admin] RBAC: Adding query filter params (role.id=05f3681d0a644607934e281f75ecd675, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=833a474f828f4349bea0d1649461c77e) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:00.220895 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23645d17-8f52-43c3-8c64-f1a871603c3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:00.222292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23645d17-8f52-43c3-8c64-f1a871603c3f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:00.228767 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 297/574] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:39:00 2022] GET /identity/v3/role_assignments?user.id=833a474f828f4349bea0d1649461c77e&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=05f3681d0a644607934e281f75ecd675 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:00.229655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23645d17-8f52-43c3-8c64-f1a871603c3f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:00.232290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07030e27-7652-4af8-8a91-674f048cdea2 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:00.295451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5acd89b-b03c-4233-8b8b-ccec9ee866cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:00.297108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acd89b-b03c-4233-8b8b-ccec9ee866cb None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:00.297797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acd89b-b03c-4233-8b8b-ccec9ee866cb None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:00.298704 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acd89b-b03c-4233-8b8b-ccec9ee866cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:00.318405 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5acd89b-b03c-4233-8b8b-ccec9ee866cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-e5acd89b-b03c-4233-8b8b-ccec9ee866cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:00.328758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:00.329990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:00.336858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:00.336858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:00.336858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:00.336858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:00.336858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:00.336858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:00.336858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:00.336858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:00.336858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:00.336858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:00.336858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:00.336858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:00.336858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 278/575] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:39:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:00.336858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5acd89b-b03c-4233-8b8b-ccec9ee866cb None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:00.357366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23645d17-8f52-43c3-8c64-f1a871603c3f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:00.401581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197bdd91-f03d-4cc5-948f-59d05c3ce237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:00.402401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bdd91-f03d-4cc5-948f-59d05c3ce237 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:00.402687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bdd91-f03d-4cc5-948f-59d05c3ce237 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:00.402967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bdd91-f03d-4cc5-948f-59d05c3ce237 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:00.410852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197bdd91-f03d-4cc5-948f-59d05c3ce237 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:00.413301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197bdd91-f03d-4cc5-948f-59d05c3ce237 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:00.436020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:00.441678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-197bdd91-f03d-4cc5-948f-59d05c3ce237 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:00.441678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:00.441678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:00.441678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:00.441678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:00.441678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:00.441678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:00.441678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:00.441678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:00.441678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:00.441678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:00.441678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:00.441678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:00.441678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:00.441678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:00.442764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:00.444363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:00.444363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:00.444363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:00.444363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:00.444363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:00.444363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:00.444363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:00.444363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:00.444363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 298/576] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:00.445600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197bdd91-f03d-4cc5-948f-59d05c3ce237 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:00.457750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:00.469201 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5acd89b-b03c-4233-8b8b-ccec9ee866cb None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:00.509957 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27fc42-657f-45c5-8453-d1e534c9fe57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:00.510634 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27fc42-657f-45c5-8453-d1e534c9fe57 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:00.510915 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27fc42-657f-45c5-8453-d1e534c9fe57 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:00.511191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27fc42-657f-45c5-8453-d1e534c9fe57 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:00.511949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27fc42-657f-45c5-8453-d1e534c9fe57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:00.512253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27fc42-657f-45c5-8453-d1e534c9fe57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:00.513668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27fc42-657f-45c5-8453-d1e534c9fe57 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:00.527246 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 279/577] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:00.528383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27fc42-657f-45c5-8453-d1e534c9fe57 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:00.571097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197bdd91-f03d-4cc5-948f-59d05c3ce237 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:00.620196 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e55ea-1de8-4b71-bf4f-84adb67e7fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:00.621295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e55ea-1de8-4b71-bf4f-84adb67e7fa8 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:00.621604 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e55ea-1de8-4b71-bf4f-84adb67e7fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:00.622112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e55ea-1de8-4b71-bf4f-84adb67e7fa8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:00.634172 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e55ea-1de8-4b71-bf4f-84adb67e7fa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-886e55ea-1de8-4b71-bf4f-84adb67e7fa8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:00.641937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:00.642907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:00.643768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:00.643768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:00.643768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:00.643768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:00.643768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:00.643768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:00.643768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:00.643768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:00.643768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:00.643768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:00.643768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:00.643768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:00.643768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 299/578] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:39:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:00.645230 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e55ea-1de8-4b71-bf4f-84adb67e7fa8 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:00.652273 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27fc42-657f-45c5-8453-d1e534c9fe57 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:00.697382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15945147-41d3-487e-a3cb-34d90e780743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:00.698071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15945147-41d3-487e-a3cb-34d90e780743 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:00.698356 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15945147-41d3-487e-a3cb-34d90e780743 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:00.698633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15945147-41d3-487e-a3cb-34d90e780743 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:00.699758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15945147-41d3-487e-a3cb-34d90e780743 None admin] RBAC: Adding query filter params (name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:00.699758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15945147-41d3-487e-a3cb-34d90e780743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:00.701109 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15945147-41d3-487e-a3cb-34d90e780743 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:00.709221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:00.714218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 280/579] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:39:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:00.715512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15945147-41d3-487e-a3cb-34d90e780743 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:00.732771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:00.772461 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e55ea-1de8-4b71-bf4f-84adb67e7fa8 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:00.809530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595de16-e062-44db-8c56-df742bd5528c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:00.810185 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595de16-e062-44db-8c56-df742bd5528c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:00.810476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595de16-e062-44db-8c56-df742bd5528c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:00.810755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595de16-e062-44db-8c56-df742bd5528c None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:00.811555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595de16-e062-44db-8c56-df742bd5528c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:00.811893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595de16-e062-44db-8c56-df742bd5528c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:00.813348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595de16-e062-44db-8c56-df742bd5528c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:00.827149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 300/580] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:00.828112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595de16-e062-44db-8c56-df742bd5528c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:00.840327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15945147-41d3-487e-a3cb-34d90e780743 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:00.884023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f3627-ee7c-4f2c-9a9b-e4947efdaeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:00.884829 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f3627-ee7c-4f2c-9a9b-e4947efdaeba None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:00.885111 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f3627-ee7c-4f2c-9a9b-e4947efdaeba None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:00.885398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f3627-ee7c-4f2c-9a9b-e4947efdaeba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:00.893469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f3627-ee7c-4f2c-9a9b-e4947efdaeba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:00.894936 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f3627-ee7c-4f2c-9a9b-e4947efdaeba None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-3c0f3627-ee7c-4f2c-9a9b-e4947efdaeba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:00.909815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:00.911529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:00.913826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:00.913826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:00.913826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:00.913826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:00.913826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:00.913826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:00.913826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:00.913826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:00.913826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:39:00.913826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:00.913826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 281/581] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:39:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:00.913826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f3627-ee7c-4f2c-9a9b-e4947efdaeba None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:00.955905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595de16-e062-44db-8c56-df742bd5528c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:00.988836 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:00.990406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:00.999433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59144d-a46e-49f7-9cf0-ee1e68421f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:01.000255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59144d-a46e-49f7-9cf0-ee1e68421f77 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:01.000536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59144d-a46e-49f7-9cf0-ee1e68421f77 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:01.000814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59144d-a46e-49f7-9cf0-ee1e68421f77 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:01.001741 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59144d-a46e-49f7-9cf0-ee1e68421f77 None admin] RBAC: Adding query filter params (name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:01.002049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59144d-a46e-49f7-9cf0-ee1e68421f77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:01.003536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59144d-a46e-49f7-9cf0-ee1e68421f77 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:01.014153 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 301/582] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:39:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:01.014153 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59144d-a46e-49f7-9cf0-ee1e68421f77 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:01.046587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f3627-ee7c-4f2c-9a9b-e4947efdaeba None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:01.092546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a045d3b-0c18-4cfa-aad2-6b4c6803827f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:01.093367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a045d3b-0c18-4cfa-aad2-6b4c6803827f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:01.093724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a045d3b-0c18-4cfa-aad2-6b4c6803827f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:01.094067 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a045d3b-0c18-4cfa-aad2-6b4c6803827f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:01.106591 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a045d3b-0c18-4cfa-aad2-6b4c6803827f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:01.108221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a045d3b-0c18-4cfa-aad2-6b4c6803827f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:01.128968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-7a045d3b-0c18-4cfa-aad2-6b4c6803827f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:01.128968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:01.128968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:01.128968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:01.128968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:01.128968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:01.128968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:01.128968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:01.128968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:01.128968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:01.128968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:01.128968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:01.128968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:01.128968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:01.128968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:01.129906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:01.131572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:01.131572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:01.131572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:01.131572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:01.131572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 282/583] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:01.132176 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a045d3b-0c18-4cfa-aad2-6b4c6803827f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:01.157007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59144d-a46e-49f7-9cf0-ee1e68421f77 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:01.200051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed7cce-f9d3-4e53-b1fd-fb687ea8c423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:01.200946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed7cce-f9d3-4e53-b1fd-fb687ea8c423 None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:01.201232 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed7cce-f9d3-4e53-b1fd-fb687ea8c423 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:01.201515 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed7cce-f9d3-4e53-b1fd-fb687ea8c423 None admin] PATH_INFO: `/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/77812b356c974d2b80102217956169db/roles/05f3681d0a644607934e281f75ecd675` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:01.223630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed7cce-f9d3-4e53-b1fd-fb687ea8c423 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc61fe15894891a9054911321f4ac3, user_id=77812b356c974d2b80102217956169db, role_id=05f3681d0a644607934e281f75ecd675)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:01.225699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed7cce-f9d3-4e53-b1fd-fb687ea8c423 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:01.243567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a045d3b-0c18-4cfa-aad2-6b4c6803827f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:01.245118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 302/584] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:39:01 2022] PUT /identity/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/77812b356c974d2b80102217956169db/roles/05f3681d0a644607934e281f75ecd675 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:01.255350 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:01.256771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed7cce-f9d3-4e53-b1fd-fb687ea8c423 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:01.295464 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc8a79-d010-4e7c-8038-4bbe8e9e51fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:01.295685 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc8a79-d010-4e7c-8038-4bbe8e9e51fc None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:01.295999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc8a79-d010-4e7c-8038-4bbe8e9e51fc None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:01.296286 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc8a79-d010-4e7c-8038-4bbe8e9e51fc None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:01.297148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc8a79-d010-4e7c-8038-4bbe8e9e51fc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:01.297466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc8a79-d010-4e7c-8038-4bbe8e9e51fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:01.298951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc8a79-d010-4e7c-8038-4bbe8e9e51fc None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:01.322778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 283/585] 10.222.0.60 () {56 vars in 1137 bytes} [Thu Jul 21 20:39:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:01.323946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc8a79-d010-4e7c-8038-4bbe8e9e51fc None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:01.328128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:01.422984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed7cce-f9d3-4e53-b1fd-fb687ea8c423 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:01.460873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc8a79-d010-4e7c-8038-4bbe8e9e51fc None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:01.469614 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed27c1e-2ec9-4763-a07d-4dc4af54eea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:01.470423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed27c1e-2ec9-4763-a07d-4dc4af54eea4 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:01.470705 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed27c1e-2ec9-4763-a07d-4dc4af54eea4 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:01.470987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed27c1e-2ec9-4763-a07d-4dc4af54eea4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:01.483453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed27c1e-2ec9-4763-a07d-4dc4af54eea4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:01.484972 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed27c1e-2ec9-4763-a07d-4dc4af54eea4 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:01.508128 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a04b90-e233-41ed-bde9-5fdda18976ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:01.509156 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a04b90-e233-41ed-bde9-5fdda18976ab None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:01.509767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a04b90-e233-41ed-bde9-5fdda18976ab None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:01.510277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a04b90-e233-41ed-bde9-5fdda18976ab None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:01.510726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-0ed27c1e-2ec9-4763-a07d-4dc4af54eea4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:01.510726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:01.510726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:01.510726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:01.510726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:01.510726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:01.510726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:01.510726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:01.510726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:01.510726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:01.510726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:01.510726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:01.510726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:01.510726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:01.510726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:01.511556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:01.512483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:01.512483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:01.512483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:01.512483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:01.512483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 303/586] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:01.513264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed27c1e-2ec9-4763-a07d-4dc4af54eea4 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:01.513944 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a04b90-e233-41ed-bde9-5fdda18976ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:01.514464 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a04b90-e233-41ed-bde9-5fdda18976ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:01.516077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a04b90-e233-41ed-bde9-5fdda18976ab None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:01.539275 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 284/587] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:01.539275 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a04b90-e233-41ed-bde9-5fdda18976ab None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:01.581017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:01.631268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:01.642088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed27c1e-2ec9-4763-a07d-4dc4af54eea4 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:01.684007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752015bf-95d4-4341-b4a8-52c47eccc16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:01.684728 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752015bf-95d4-4341-b4a8-52c47eccc16f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:01.685009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752015bf-95d4-4341-b4a8-52c47eccc16f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:01.685300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752015bf-95d4-4341-b4a8-52c47eccc16f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:01.688042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a04b90-e233-41ed-bde9-5fdda18976ab None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:01.698264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752015bf-95d4-4341-b4a8-52c47eccc16f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:01.699810 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752015bf-95d4-4341-b4a8-52c47eccc16f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:01.721174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-752015bf-95d4-4341-b4a8-52c47eccc16f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:01.721174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:01.721174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:01.721174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:01.721174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:01.721174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:01.721174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:01.721174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:01.721174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:01.721174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:01.721174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:01.721174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:01.721174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:01.721174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:01.721174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:01.722125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:01.723003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:01.723003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:01.723003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:01.723003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:01.727955 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 304/588] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:01.729049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752015bf-95d4-4341-b4a8-52c47eccc16f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:01.732574 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf42b8f-097a-4738-974e-518bf68624f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:01.734344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf42b8f-097a-4738-974e-518bf68624f8 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:01.734821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf42b8f-097a-4738-974e-518bf68624f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:01.735285 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf42b8f-097a-4738-974e-518bf68624f8 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:01.736242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf42b8f-097a-4738-974e-518bf68624f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:01.736767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf42b8f-097a-4738-974e-518bf68624f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:01.738364 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf42b8f-097a-4738-974e-518bf68624f8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:01.755023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 285/589] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:01.756891 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf42b8f-097a-4738-974e-518bf68624f8 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:01.872944 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:01.875305 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752015bf-95d4-4341-b4a8-52c47eccc16f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:01.889027 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf42b8f-097a-4738-974e-518bf68624f8 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:01.923943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24016fa1-8803-4872-bc0d-6651e80ff9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:01.925252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24016fa1-8803-4872-bc0d-6651e80ff9f0 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:01.925789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24016fa1-8803-4872-bc0d-6651e80ff9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:01.926360 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24016fa1-8803-4872-bc0d-6651e80ff9f0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:01.930363 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d024b8-7791-41c7-b1af-1e0c83ae846f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:01.930881 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d024b8-7791-41c7-b1af-1e0c83ae846f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:01.931266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d024b8-7791-41c7-b1af-1e0c83ae846f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:01.931651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d024b8-7791-41c7-b1af-1e0c83ae846f None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:01.932529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d024b8-7791-41c7-b1af-1e0c83ae846f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:01.932938 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d024b8-7791-41c7-b1af-1e0c83ae846f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:01.934574 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d024b8-7791-41c7-b1af-1e0c83ae846f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:01.936524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24016fa1-8803-4872-bc0d-6651e80ff9f0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:01.938292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24016fa1-8803-4872-bc0d-6651e80ff9f0 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-24016fa1-8803-4872-bc0d-6651e80ff9f0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:01.955637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:01.956743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:01.957662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:01.957662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:01.957662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:01.957662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:01.957662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:01.957662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:01.957662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:01.957662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:01.957662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 20:39:01.957662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:01.958209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 286/590] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:01.958209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d024b8-7791-41c7-b1af-1e0c83ae846f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:01.965332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 305/591] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:39:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:01.966617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24016fa1-8803-4872-bc0d-6651e80ff9f0 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:01.970686 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:02.078568 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d024b8-7791-41c7-b1af-1e0c83ae846f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:02.086876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24016fa1-8803-4872-bc0d-6651e80ff9f0 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:02.117165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c322e-a3c1-45b9-aaf5-8f6de3451e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:02.118046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c322e-a3c1-45b9-aaf5-8f6de3451e55 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:02.118596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c322e-a3c1-45b9-aaf5-8f6de3451e55 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:02.118804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c322e-a3c1-45b9-aaf5-8f6de3451e55 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:02.126881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95ee6d-0bd3-4963-82ab-6224ec21b63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:02.127944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95ee6d-0bd3-4963-82ab-6224ec21b63c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:02.128410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95ee6d-0bd3-4963-82ab-6224ec21b63c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:02.128805 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c322e-a3c1-45b9-aaf5-8f6de3451e55 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:02.129026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95ee6d-0bd3-4963-82ab-6224ec21b63c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:02.131029 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c322e-a3c1-45b9-aaf5-8f6de3451e55 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:02.138086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95ee6d-0bd3-4963-82ab-6224ec21b63c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:02.139833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95ee6d-0bd3-4963-82ab-6224ec21b63c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-3b8c322e-a3c1-45b9-aaf5-8f6de3451e55 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:02.148293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:02.149483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:02.150612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:02.150612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:02.150612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:02.150612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:02.150612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:02.150612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:02.150612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:02.150612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:02.150612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 20:39:02.150612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:02.156042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 287/592] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:39:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:02.157030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c322e-a3c1-45b9-aaf5-8f6de3451e55 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:02.159369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-7f95ee6d-0bd3-4963-82ab-6224ec21b63c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:02.159369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:02.159369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:02.159369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:02.159369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:02.159369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:02.159369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:02.159369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:02.159369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:02.159369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:02.159369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:02.159369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:02.159369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:02.159369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:02.159369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:02.160437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:02.161295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:02.161295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:02.161295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:02.161295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:02.161295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:02.161295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:02.161295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:02.161295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:02.163892 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 306/593] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:02.164937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95ee6d-0bd3-4963-82ab-6224ec21b63c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:02.174413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:02.239870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:02.282901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c322e-a3c1-45b9-aaf5-8f6de3451e55 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:02.289747 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95ee6d-0bd3-4963-82ab-6224ec21b63c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:02.324894 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d62b14-de34-47c5-aab9-8c6e882e7d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:02.325727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d62b14-de34-47c5-aab9-8c6e882e7d62 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:02.325992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d62b14-de34-47c5-aab9-8c6e882e7d62 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:02.326264 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d62b14-de34-47c5-aab9-8c6e882e7d62 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:02.327120 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d62b14-de34-47c5-aab9-8c6e882e7d62 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:02.327430 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d62b14-de34-47c5-aab9-8c6e882e7d62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:02.328998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d62b14-de34-47c5-aab9-8c6e882e7d62 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:02.330117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cec111-f628-4f2d-97e1-ce51e8970472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:02.331146 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cec111-f628-4f2d-97e1-ce51e8970472 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:02.331663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cec111-f628-4f2d-97e1-ce51e8970472 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:02.332224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cec111-f628-4f2d-97e1-ce51e8970472 None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:02.333317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cec111-f628-4f2d-97e1-ce51e8970472 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:02.334350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cec111-f628-4f2d-97e1-ce51e8970472 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:02.336268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cec111-f628-4f2d-97e1-ce51e8970472 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:02.352153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 288/594] 10.222.0.60 () {56 vars in 1137 bytes} [Thu Jul 21 20:39:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:02.353963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d62b14-de34-47c5-aab9-8c6e882e7d62 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:02.357753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 307/595] 10.222.0.60 () {56 vars in 1139 bytes} [Thu Jul 21 20:39:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:02.362308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cec111-f628-4f2d-97e1-ce51e8970472 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:02.451442 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:02.492900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cec111-f628-4f2d-97e1-ce51e8970472 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:02.494639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d62b14-de34-47c5-aab9-8c6e882e7d62 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:02.520255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:02.533314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5d5d4-26a6-4238-b96a-83850acf9ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:02.534114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5d5d4-26a6-4238-b96a-83850acf9ca3 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:02.534350 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b685ed-5717-4e53-9394-b4decbedb3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:02.534469 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5d5d4-26a6-4238-b96a-83850acf9ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:02.534658 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5d5d4-26a6-4238-b96a-83850acf9ca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:02.535686 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b685ed-5717-4e53-9394-b4decbedb3d7 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:02.536223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b685ed-5717-4e53-9394-b4decbedb3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:02.536913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b685ed-5717-4e53-9394-b4decbedb3d7 None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:02.538410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b685ed-5717-4e53-9394-b4decbedb3d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:02.538958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b685ed-5717-4e53-9394-b4decbedb3d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:02.540664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b685ed-5717-4e53-9394-b4decbedb3d7 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:02.545980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5d5d4-26a6-4238-b96a-83850acf9ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:02.547442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5d5d4-26a6-4238-b96a-83850acf9ca3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:02.556935 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 289/596] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:02.558592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b685ed-5717-4e53-9394-b4decbedb3d7 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:02.567150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-8fa5d5d4-26a6-4238-b96a-83850acf9ca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:02.567150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:02.567150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:02.567150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:02.567150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:02.567150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:02.567150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:02.567150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:02.567150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:02.567150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:02.567150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:02.567150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:02.567150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:02.567150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:02.567150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:02.568386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:02.569404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:02.569404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:02.569404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:02.569404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:02.573317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 308/597] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:02.574637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a698a0-1867-450e-b27b-16aa650e2d6b None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:02.574928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a698a0-1867-450e-b27b-16aa650e2d6b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:02.575243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a698a0-1867-450e-b27b-16aa650e2d6b None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:02.580918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 309/598] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:02.581802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a698a0-1867-450e-b27b-16aa650e2d6b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:02.680329 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b685ed-5717-4e53-9394-b4decbedb3d7 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:02.710247 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:02.715059 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a698a0-1867-450e-b27b-16aa650e2d6b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:02.718646 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a3925-fde8-475d-95f0-d1af9fab5aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:02.719447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a3925-fde8-475d-95f0-d1af9fab5aa5 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:02.719743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a3925-fde8-475d-95f0-d1af9fab5aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:02.720025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a3925-fde8-475d-95f0-d1af9fab5aa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:02.731324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a3925-fde8-475d-95f0-d1af9fab5aa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:02.732829 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a3925-fde8-475d-95f0-d1af9fab5aa5 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:02.750398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-732a3925-fde8-475d-95f0-d1af9fab5aa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:02.750398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:02.750398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:02.750398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:02.750398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:02.750398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:02.750398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:02.750398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:02.750398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:02.750398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:02.750398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:02.750398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:02.750398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:02.750398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:02.750398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:02.751380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:02.752282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:02.752282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:02.752282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:02.752282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:02.752282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 290/599] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:02.752568 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a3925-fde8-475d-95f0-d1af9fab5aa5 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:02.765542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91838310-d0b0-4d86-af0a-ad606f1960fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:02.767336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91838310-d0b0-4d86-af0a-ad606f1960fc None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:02.768123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91838310-d0b0-4d86-af0a-ad606f1960fc None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:02.768850 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91838310-d0b0-4d86-af0a-ad606f1960fc None admin] PATH_INFO: `/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/a8eef7746fc4417e8d4de0b052f3eaa0/roles/05f3681d0a644607934e281f75ecd675` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:02.791003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91838310-d0b0-4d86-af0a-ad606f1960fc None admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc61fe15894891a9054911321f4ac3, user_id=a8eef7746fc4417e8d4de0b052f3eaa0, role_id=05f3681d0a644607934e281f75ecd675)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:02.791003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91838310-d0b0-4d86-af0a-ad606f1960fc None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:02.808884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 310/600] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:39:02 2022] PUT /identity/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/a8eef7746fc4417e8d4de0b052f3eaa0/roles/05f3681d0a644607934e281f75ecd675 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:02.810610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b2178-c02c-4e04-a9cf-4d61e0cceee9 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:02.811089 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b2178-c02c-4e04-a9cf-4d61e0cceee9 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:02.811554 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b2178-c02c-4e04-a9cf-4d61e0cceee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:02.830967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-5d4b2178-c02c-4e04-a9cf-4d61e0cceee9 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:02.832579 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-5d4b2178-c02c-4e04-a9cf-4d61e0cceee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:02.860215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:02.914049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a3925-fde8-475d-95f0-d1af9fab5aa5 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:02.942131 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 311/601] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:39:02.944820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b2178-c02c-4e04-a9cf-4d61e0cceee9 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:02.966621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ed486-79e6-4c32-bc76-ea66dbd3e4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:02.966621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed486-79e6-4c32-bc76-ea66dbd3e4a7 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:02.966621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed486-79e6-4c32-bc76-ea66dbd3e4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:02.966621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed486-79e6-4c32-bc76-ea66dbd3e4a7 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:02.967043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ed486-79e6-4c32-bc76-ea66dbd3e4a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:02.967043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ed486-79e6-4c32-bc76-ea66dbd3e4a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:02.967043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ed486-79e6-4c32-bc76-ea66dbd3e4a7 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:02.983657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 291/602] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:02.990885 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ed486-79e6-4c32-bc76-ea66dbd3e4a7 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:03.056381 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:03.102022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b2178-c02c-4e04-a9cf-4d61e0cceee9 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:03.141929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d7c58-c558-4cee-890f-c6238e110d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:03.142944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d7c58-c558-4cee-890f-c6238e110d38 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:03.143432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d7c58-c558-4cee-890f-c6238e110d38 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:03.143934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d7c58-c558-4cee-890f-c6238e110d38 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:03.144967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d7c58-c558-4cee-890f-c6238e110d38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:03.145497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d7c58-c558-4cee-890f-c6238e110d38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:03.147830 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d7c58-c558-4cee-890f-c6238e110d38 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:03.150404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ed486-79e6-4c32-bc76-ea66dbd3e4a7 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:03.166651 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 312/603] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:03.168001 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d7c58-c558-4cee-890f-c6238e110d38 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:03.186860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:03.200067 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d17bb-53ce-48aa-9b28-7f099e68d43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:03.201240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d17bb-53ce-48aa-9b28-7f099e68d43b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:03.201554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d17bb-53ce-48aa-9b28-7f099e68d43b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:03.202207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d17bb-53ce-48aa-9b28-7f099e68d43b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:03.213401 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d17bb-53ce-48aa-9b28-7f099e68d43b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:03.215507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d17bb-53ce-48aa-9b28-7f099e68d43b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:03.231542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-842d17bb-53ce-48aa-9b28-7f099e68d43b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:03.231542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:03.231542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:03.231542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:03.231542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:03.231542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:03.231542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:03.231542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:03.231542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:03.231542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:03.231542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:03.231542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:03.231542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:03.231542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:03.231542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:03.232518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:03.233387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:03.233387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:03.233387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:03.233387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:03.233387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:03.233387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:03.233387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:03.233387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:03.236018 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 292/604] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:03.241446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d17bb-53ce-48aa-9b28-7f099e68d43b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:03.311210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d7c58-c558-4cee-890f-c6238e110d38 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:03.359576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294cc6c-dff7-43d1-9724-18958ab21f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:03.360441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294cc6c-dff7-43d1-9724-18958ab21f7b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:03.360726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294cc6c-dff7-43d1-9724-18958ab21f7b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:03.361001 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294cc6c-dff7-43d1-9724-18958ab21f7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:03.362059 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:03.367585 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d17bb-53ce-48aa-9b28-7f099e68d43b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:03.378281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3294cc6c-dff7-43d1-9724-18958ab21f7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-3294cc6c-dff7-43d1-9724-18958ab21f7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:03.389182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:03.390138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:03.390958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:03.390958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:03.390958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:03.390958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:03.390958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:03.390958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:03.390958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:03.390958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:03.390958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:03.390958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:03.390958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:03.390958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:03.393886 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 313/605] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:39:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:03.395548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294cc6c-dff7-43d1-9724-18958ab21f7b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:03.422594 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f7cb8-3c82-4b93-b8e2-28926ed0b1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:03.423984 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f7cb8-3c82-4b93-b8e2-28926ed0b1c7 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:03.424479 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f7cb8-3c82-4b93-b8e2-28926ed0b1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:03.424998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f7cb8-3c82-4b93-b8e2-28926ed0b1c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:03.433824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f7cb8-3c82-4b93-b8e2-28926ed0b1c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:03.435903 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f7cb8-3c82-4b93-b8e2-28926ed0b1c7 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:03.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-975f7cb8-3c82-4b93-b8e2-28926ed0b1c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:03.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:03.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:03.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:03.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:03.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:03.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:03.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:03.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:03.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:03.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:03.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:03.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:03.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:03.452560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:03.453484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:03.454380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:03.454380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:03.454380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:03.454380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:03.454380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:03.454380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:03.454380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:03.454380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:03.456246 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 293/606] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:03.457236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f7cb8-3c82-4b93-b8e2-28926ed0b1c7 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:03.501659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:03.530940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294cc6c-dff7-43d1-9724-18958ab21f7b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:03.578093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777deea-c2ac-4ffd-bf50-6007d451a069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:03.578622 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f7cb8-3c82-4b93-b8e2-28926ed0b1c7 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:03.578740 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777deea-c2ac-4ffd-bf50-6007d451a069 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:03.579018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777deea-c2ac-4ffd-bf50-6007d451a069 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:03.579292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777deea-c2ac-4ffd-bf50-6007d451a069 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:03.580124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777deea-c2ac-4ffd-bf50-6007d451a069 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:03.580430 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777deea-c2ac-4ffd-bf50-6007d451a069 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:03.581991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777deea-c2ac-4ffd-bf50-6007d451a069 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:03.597298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 314/607] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:03.598662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1f89b-f053-462a-8d8f-f6faaa10b55e None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:03.598950 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1f89b-f053-462a-8d8f-f6faaa10b55e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:03.599798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1f89b-f053-462a-8d8f-f6faaa10b55e None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:03.605588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 315/608] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:03.606463 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1f89b-f053-462a-8d8f-f6faaa10b55e None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:03.626717 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b68a0b-ca38-4295-903d-83d1772ae6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:03.627801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b68a0b-ca38-4295-903d-83d1772ae6dc None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:03.628235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b68a0b-ca38-4295-903d-83d1772ae6dc None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:03.628660 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b68a0b-ca38-4295-903d-83d1772ae6dc None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:03.629562 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b68a0b-ca38-4295-903d-83d1772ae6dc None admin] RBAC: Adding query filter params (name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:03.630015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b68a0b-ca38-4295-903d-83d1772ae6dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:03.632032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b68a0b-ca38-4295-903d-83d1772ae6dc None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:03.641644 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 294/609] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:39:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:03.642755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b68a0b-ca38-4295-903d-83d1772ae6dc None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:03.668065 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:03.738902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1f89b-f053-462a-8d8f-f6faaa10b55e None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:03.771248 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b68a0b-ca38-4295-903d-83d1772ae6dc None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:03.779862 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8ba3c-ccaa-43e9-a6d1-184d76891981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:03.780720 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8ba3c-ccaa-43e9-a6d1-184d76891981 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:03.780802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8ba3c-ccaa-43e9-a6d1-184d76891981 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:03.781065 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8ba3c-ccaa-43e9-a6d1-184d76891981 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:03.781988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8ba3c-ccaa-43e9-a6d1-184d76891981 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:03.782299 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8ba3c-ccaa-43e9-a6d1-184d76891981 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:03.783769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8ba3c-ccaa-43e9-a6d1-184d76891981 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:03.785851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:03.800530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 316/610] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:03.801915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39f1d7-6396-4e04-aeb4-dfb0312ea322 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:03.802216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39f1d7-6396-4e04-aeb4-dfb0312ea322 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:03.802632 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39f1d7-6396-4e04-aeb4-dfb0312ea322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:03.814851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fdb06-df1f-43d8-b86c-6125db25ef0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:03.815776 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fdb06-df1f-43d8-b86c-6125db25ef0e None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:03.816069 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fdb06-df1f-43d8-b86c-6125db25ef0e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:03.816341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fdb06-df1f-43d8-b86c-6125db25ef0e None admin] PATH_INFO: `/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/1d596de177e24ca5a254bda6644a07da/roles/05f3681d0a644607934e281f75ecd675` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:03.822918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-2b39f1d7-6396-4e04-aeb4-dfb0312ea322 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:03.824163 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-2b39f1d7-6396-4e04-aeb4-dfb0312ea322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:03.839256 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11fdb06-df1f-43d8-b86c-6125db25ef0e None admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc61fe15894891a9054911321f4ac3, user_id=1d596de177e24ca5a254bda6644a07da, role_id=05f3681d0a644607934e281f75ecd675)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:03.840778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11fdb06-df1f-43d8-b86c-6125db25ef0e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:03.857923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 295/611] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:39:03 2022] PUT /identity/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/1d596de177e24ca5a254bda6644a07da/roles/05f3681d0a644607934e281f75ecd675 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:03.859114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fdb06-df1f-43d8-b86c-6125db25ef0e None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:03.924430 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 317/612] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:39:03.925692 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39f1d7-6396-4e04-aeb4-dfb0312ea322 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:03.966884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:04.008760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fdb06-df1f-43d8-b86c-6125db25ef0e None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:04.059502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28774747-c6fa-4b9a-98d0-2a8e5d2e9d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:04.060747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28774747-c6fa-4b9a-98d0-2a8e5d2e9d3d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:04.062692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28774747-c6fa-4b9a-98d0-2a8e5d2e9d3d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:04.063223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28774747-c6fa-4b9a-98d0-2a8e5d2e9d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:04.071031 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39f1d7-6396-4e04-aeb4-dfb0312ea322 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:04.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28774747-c6fa-4b9a-98d0-2a8e5d2e9d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:04.080103 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28774747-c6fa-4b9a-98d0-2a8e5d2e9d3d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:04.103205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-28774747-c6fa-4b9a-98d0-2a8e5d2e9d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:04.103205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:04.103205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:04.103205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:04.103205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:04.103205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:04.103205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:04.103205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:04.103205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:04.103205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:04.103205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:04.103205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:04.103205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:04.103205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:04.103205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:04.104692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:04.105572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:04.105572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:04.105572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:04.105572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:04.105572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 296/613] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:04.105572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28774747-c6fa-4b9a-98d0-2a8e5d2e9d3d None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:04.106563 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:04.131324 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501766ec-aae7-4755-87eb-2b3a0a07fdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:04.132868 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501766ec-aae7-4755-87eb-2b3a0a07fdae None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:04.133400 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501766ec-aae7-4755-87eb-2b3a0a07fdae None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:04.134227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501766ec-aae7-4755-87eb-2b3a0a07fdae None admin] PATH_INFO: `/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/33aff48933844401b86eb808937e35c4/roles/05f3681d0a644607934e281f75ecd675` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:04.156474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501766ec-aae7-4755-87eb-2b3a0a07fdae None admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc61fe15894891a9054911321f4ac3, user_id=33aff48933844401b86eb808937e35c4, role_id=05f3681d0a644607934e281f75ecd675)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:04.158277 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501766ec-aae7-4755-87eb-2b3a0a07fdae None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:04.177041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 318/614] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:39:03 2022] PUT /identity/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/33aff48933844401b86eb808937e35c4/roles/05f3681d0a644607934e281f75ecd675 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:04.178911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501766ec-aae7-4755-87eb-2b3a0a07fdae None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:04.254608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28774747-c6fa-4b9a-98d0-2a8e5d2e9d3d None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:04.290171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:04.299779 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688a3af-2cb6-4143-aede-05432eb3b4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:04.300987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688a3af-2cb6-4143-aede-05432eb3b4bc None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:04.301511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688a3af-2cb6-4143-aede-05432eb3b4bc None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:04.301996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688a3af-2cb6-4143-aede-05432eb3b4bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:04.314876 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688a3af-2cb6-4143-aede-05432eb3b4bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-4688a3af-2cb6-4143-aede-05432eb3b4bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:04.329873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:04.331676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:04.333762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:04.333762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:04.333762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:04.333762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:04.333762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:04.333762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:04.333762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:04.333762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:04.333762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:04.333762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:04.333762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:04.333762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:04.336752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 297/615] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:39:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:04.343165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688a3af-2cb6-4143-aede-05432eb3b4bc None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:04.426063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501766ec-aae7-4755-87eb-2b3a0a07fdae None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:04.469530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25446b62-3b10-4bc1-b17e-4dcb468c0220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:04.470205 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25446b62-3b10-4bc1-b17e-4dcb468c0220 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:04.470487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25446b62-3b10-4bc1-b17e-4dcb468c0220 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:04.470765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25446b62-3b10-4bc1-b17e-4dcb468c0220 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:04.471914 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25446b62-3b10-4bc1-b17e-4dcb468c0220 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:04.471914 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25446b62-3b10-4bc1-b17e-4dcb468c0220 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:04.473306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25446b62-3b10-4bc1-b17e-4dcb468c0220 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:04.483443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688a3af-2cb6-4143-aede-05432eb3b4bc None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:04.491746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 319/616] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:04.496144 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25446b62-3b10-4bc1-b17e-4dcb468c0220 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:04.533322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e121bbc-5155-44fc-bd10-488fdd64d123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:04.534529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e121bbc-5155-44fc-bd10-488fdd64d123 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:04.535121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e121bbc-5155-44fc-bd10-488fdd64d123 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:04.535625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e121bbc-5155-44fc-bd10-488fdd64d123 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:04.537155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e121bbc-5155-44fc-bd10-488fdd64d123 None admin] RBAC: Adding query filter params (name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:04.537678 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e121bbc-5155-44fc-bd10-488fdd64d123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:04.539243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:04.540698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e121bbc-5155-44fc-bd10-488fdd64d123 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:04.553349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 298/617] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:39:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:04.560856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e121bbc-5155-44fc-bd10-488fdd64d123 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:04.638226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25446b62-3b10-4bc1-b17e-4dcb468c0220 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:04.641530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:04.705509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e121bbc-5155-44fc-bd10-488fdd64d123 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:04.720501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcfd52-22ca-4f37-a7b9-8e2dbc273343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:04.721347 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcfd52-22ca-4f37-a7b9-8e2dbc273343 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:04.721630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcfd52-22ca-4f37-a7b9-8e2dbc273343 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:04.721907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcfd52-22ca-4f37-a7b9-8e2dbc273343 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:04.736203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcfd52-22ca-4f37-a7b9-8e2dbc273343 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:04.737677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcfd52-22ca-4f37-a7b9-8e2dbc273343 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:04.753317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba3687-67e9-4bc2-a7a0-49ba2670732c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:04.754484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba3687-67e9-4bc2-a7a0-49ba2670732c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:04.755008 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba3687-67e9-4bc2-a7a0-49ba2670732c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:04.755617 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba3687-67e9-4bc2-a7a0-49ba2670732c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-2cbcfd52-22ca-4f37-a7b9-8e2dbc273343 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:04.774781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:04.775838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:04.776792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:04.776792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:04.776792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:04.776792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:04.776792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:04.776792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:04.776792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:04.776792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:04.776792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:39:04.776792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:04.783892 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 320/618] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:39:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:04.785507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba3687-67e9-4bc2-a7a0-49ba2670732c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:04.787666 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba3687-67e9-4bc2-a7a0-49ba2670732c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:04.795739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcfd52-22ca-4f37-a7b9-8e2dbc273343 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:04.821145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-72ba3687-67e9-4bc2-a7a0-49ba2670732c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:04.821145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:04.821145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:04.821145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:04.821145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:04.821145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:04.821145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:04.821145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:04.821145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:04.821145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:04.821145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:04.821145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:04.821145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:04.821145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:04.821145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:04.822262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:04.823808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:04.823808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:04.823808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:04.823808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:04.823808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 299/619] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:04.830500 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba3687-67e9-4bc2-a7a0-49ba2670732c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:04.968833 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:04.976814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:04.978688 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba3687-67e9-4bc2-a7a0-49ba2670732c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:04.999294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcfd52-22ca-4f37-a7b9-8e2dbc273343 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:05.017846 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c16f0-93ad-4f5c-9dcb-98d8b73f004d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:05.018527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c16f0-93ad-4f5c-9dcb-98d8b73f004d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:05.018813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c16f0-93ad-4f5c-9dcb-98d8b73f004d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:05.019093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c16f0-93ad-4f5c-9dcb-98d8b73f004d None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:05.019868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c16f0-93ad-4f5c-9dcb-98d8b73f004d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:05.020178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c16f0-93ad-4f5c-9dcb-98d8b73f004d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:05.021743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c16f0-93ad-4f5c-9dcb-98d8b73f004d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:05.035511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 300/620] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:05.044342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c16f0-93ad-4f5c-9dcb-98d8b73f004d None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:05.069988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b666e-c2e2-471e-9474-5cd2433262d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:05.070680 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b666e-c2e2-471e-9474-5cd2433262d6 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:05.071224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b666e-c2e2-471e-9474-5cd2433262d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:05.071768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b666e-c2e2-471e-9474-5cd2433262d6 None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:05.072384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b666e-c2e2-471e-9474-5cd2433262d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:05.072694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b666e-c2e2-471e-9474-5cd2433262d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:05.074142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b666e-c2e2-471e-9474-5cd2433262d6 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:05.098649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 321/621] 10.222.0.60 () {56 vars in 1135 bytes} [Thu Jul 21 20:39:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:05.100847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f56fc-465f-4761-986a-e8f904f57ea1 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:05.101133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f56fc-465f-4761-986a-e8f904f57ea1 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:05.101411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f56fc-465f-4761-986a-e8f904f57ea1 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:05.103069 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 322/622] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:05.103812 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f56fc-465f-4761-986a-e8f904f57ea1 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:05.174177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c16f0-93ad-4f5c-9dcb-98d8b73f004d None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:05.228019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66577b7a-0b70-4268-94ec-21208021673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:05.228858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66577b7a-0b70-4268-94ec-21208021673a None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:05.229473 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66577b7a-0b70-4268-94ec-21208021673a None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:05.229814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66577b7a-0b70-4268-94ec-21208021673a None admin] PATH_INFO: `/v3/users/nova` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:05.231110 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f56fc-465f-4761-986a-e8f904f57ea1 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:05.238930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66577b7a-0b70-4268-94ec-21208021673a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:05.241909 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66577b7a-0b70-4268-94ec-21208021673a None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-66577b7a-0b70-4268-94ec-21208021673a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:05.258871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:05.260313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:05.261244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:05.261244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:05.261244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:05.261244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:05.261244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:05.261244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:05.261244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:05.261244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:05.261244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:39:05.261244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:05.261244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 301/623] 10.222.0.60 () {56 vars in 1093 bytes} [Thu Jul 21 20:39:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:05.263858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fa921-024b-47cf-82b4-d939b38367fe None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:05.263858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fa921-024b-47cf-82b4-d939b38367fe None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:05.263858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fa921-024b-47cf-82b4-d939b38367fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:05.269605 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d14bf3-3c71-4d70-a545-cc1c2a3ee7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:05.270412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d14bf3-3c71-4d70-a545-cc1c2a3ee7ca None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:05.270696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d14bf3-3c71-4d70-a545-cc1c2a3ee7ca None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:05.271087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d14bf3-3c71-4d70-a545-cc1c2a3ee7ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:05.283256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d14bf3-3c71-4d70-a545-cc1c2a3ee7ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:05.284753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d14bf3-3c71-4d70-a545-cc1c2a3ee7ca None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:05.287780 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-4f2fa921-024b-47cf-82b4-d939b38367fe None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:05.290212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-4f2fa921-024b-47cf-82b4-d939b38367fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:05.303892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:05.304897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-76d14bf3-3c71-4d70-a545-cc1c2a3ee7ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:05.304897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:05.304897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:05.304897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:05.304897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:05.304897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:05.304897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:05.304897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:05.304897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:05.304897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:05.304897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:05.304897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:05.304897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:05.304897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:05.304897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:05.305918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:05.307589 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:05.307589 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:05.307589 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:05.307589 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:05.311163 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 323/624] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:05.312175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d14bf3-3c71-4d70-a545-cc1c2a3ee7ca None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:05.315963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:05.384026 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 302/625] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:39:05.385351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fa921-024b-47cf-82b4-d939b38367fe None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:05.434377 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d14bf3-3c71-4d70-a545-cc1c2a3ee7ca None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:05.471957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2117979-747a-45c6-8565-243a2df9a12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:05.472644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2117979-747a-45c6-8565-243a2df9a12f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:05.472934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2117979-747a-45c6-8565-243a2df9a12f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:05.473216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2117979-747a-45c6-8565-243a2df9a12f None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:05.474124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2117979-747a-45c6-8565-243a2df9a12f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:05.474608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2117979-747a-45c6-8565-243a2df9a12f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:05.476426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2117979-747a-45c6-8565-243a2df9a12f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:05.496300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 324/626] 10.222.0.60 () {56 vars in 1133 bytes} [Thu Jul 21 20:39:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:05.497160 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2117979-747a-45c6-8565-243a2df9a12f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:05.517527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fa921-024b-47cf-82b4-d939b38367fe None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:05.559216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928a995-bfbe-41e4-a4d6-554a31f44f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:05.560631 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928a995-bfbe-41e4-a4d6-554a31f44f5b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:05.561094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928a995-bfbe-41e4-a4d6-554a31f44f5b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:05.561553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928a995-bfbe-41e4-a4d6-554a31f44f5b None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:05.563142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:05.563638 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928a995-bfbe-41e4-a4d6-554a31f44f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:05.565441 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928a995-bfbe-41e4-a4d6-554a31f44f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:05.570051 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928a995-bfbe-41e4-a4d6-554a31f44f5b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:05.584544 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 303/627] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:05.585866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928a995-bfbe-41e4-a4d6-554a31f44f5b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:05.611047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2117979-747a-45c6-8565-243a2df9a12f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:05.640131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:05.671224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27aaf7-b7fc-4b46-a6cf-707d5eefda8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:05.672080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27aaf7-b7fc-4b46-a6cf-707d5eefda8c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:05.672365 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27aaf7-b7fc-4b46-a6cf-707d5eefda8c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:05.672894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27aaf7-b7fc-4b46-a6cf-707d5eefda8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:05.686007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27aaf7-b7fc-4b46-a6cf-707d5eefda8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-9f27aaf7-b7fc-4b46-a6cf-707d5eefda8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:05.697799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:05.698977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:05.699943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:05.699943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:05.699943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:05.699943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:05.699943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:05.699943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:05.699943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:05.699943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:05.699943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:05.699943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:05.699943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:05.699943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:05.699943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 325/628] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:39:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:05.699943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27aaf7-b7fc-4b46-a6cf-707d5eefda8c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:05.726690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928a995-bfbe-41e4-a4d6-554a31f44f5b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:05.782824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f82851-c497-4164-89f4-36f193a792cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:05.783558 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f82851-c497-4164-89f4-36f193a792cd None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:05.784042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f82851-c497-4164-89f4-36f193a792cd None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:05.784321 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f82851-c497-4164-89f4-36f193a792cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:05.840466 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27aaf7-b7fc-4b46-a6cf-707d5eefda8c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:05.868954 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:05.874553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f82851-c497-4164-89f4-36f193a792cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:05.878249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f82851-c497-4164-89f4-36f193a792cd None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:05.887198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc4317-ef0b-43ca-a6fb-3c16d7a76098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:05.888309 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc4317-ef0b-43ca-a6fb-3c16d7a76098 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:05.888807 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc4317-ef0b-43ca-a6fb-3c16d7a76098 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:05.889226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc4317-ef0b-43ca-a6fb-3c16d7a76098 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:05.889410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:05.899980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc4317-ef0b-43ca-a6fb-3c16d7a76098 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:05.902026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc4317-ef0b-43ca-a6fb-3c16d7a76098 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:05.920560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-7fcc4317-ef0b-43ca-a6fb-3c16d7a76098 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:05.920560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:05.920560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:05.920560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:05.920560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:05.920560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:05.920560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:05.920560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:05.920560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:05.920560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:05.920560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:05.920560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:05.920560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:05.920560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:05.920560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:05.921523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:05.922410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:05.922410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:05.922410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:05.922410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:05.922410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:05.922410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:05.922410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:05.922410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:05.925079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 326/629] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:05.926141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc4317-ef0b-43ca-a6fb-3c16d7a76098 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:06.011610 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 304/630] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:39:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:39:06.012898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f82851-c497-4164-89f4-36f193a792cd None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:06.061423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc4317-ef0b-43ca-a6fb-3c16d7a76098 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:06.106715 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507f107-f6d3-4b51-b8ec-2deb1f9e461a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:06.108249 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507f107-f6d3-4b51-b8ec-2deb1f9e461a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:06.108752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507f107-f6d3-4b51-b8ec-2deb1f9e461a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:06.109226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507f107-f6d3-4b51-b8ec-2deb1f9e461a None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:06.110180 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6507f107-f6d3-4b51-b8ec-2deb1f9e461a None admin] RBAC: Adding query filter params (name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:06.110683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6507f107-f6d3-4b51-b8ec-2deb1f9e461a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:06.112324 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6507f107-f6d3-4b51-b8ec-2deb1f9e461a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:06.122673 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 327/631] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:39:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:06.123952 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507f107-f6d3-4b51-b8ec-2deb1f9e461a None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:06.154681 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f82851-c497-4164-89f4-36f193a792cd None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:06.167060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:06.178514 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:06.205265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180efa9-d80c-4cf4-b065-9706903f666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:06.206314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180efa9-d80c-4cf4-b065-9706903f666c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:06.206817 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180efa9-d80c-4cf4-b065-9706903f666c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:06.207291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180efa9-d80c-4cf4-b065-9706903f666c None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:06.208319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180efa9-d80c-4cf4-b065-9706903f666c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:06.208878 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180efa9-d80c-4cf4-b065-9706903f666c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:06.210541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180efa9-d80c-4cf4-b065-9706903f666c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:06.225193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 305/632] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:06.226994 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180efa9-d80c-4cf4-b065-9706903f666c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:06.258032 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507f107-f6d3-4b51-b8ec-2deb1f9e461a None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:06.296593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b656d4-f714-45a3-bdf2-961d02a4c3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:06.297386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b656d4-f714-45a3-bdf2-961d02a4c3fe None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:06.297671 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b656d4-f714-45a3-bdf2-961d02a4c3fe None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:06.297946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b656d4-f714-45a3-bdf2-961d02a4c3fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:06.312220 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b656d4-f714-45a3-bdf2-961d02a4c3fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:06.315638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b656d4-f714-45a3-bdf2-961d02a4c3fe None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:06.335693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-c4b656d4-f714-45a3-bdf2-961d02a4c3fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:06.335693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:06.335693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:06.335693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:06.335693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:06.335693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:06.335693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:06.335693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:06.335693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:06.335693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:06.335693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:06.335693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:06.335693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:06.335693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:06.335693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:06.336660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:06.342348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:06.342348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:06.342348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:06.342348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:06.342348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 328/633] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:06.342348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b656d4-f714-45a3-bdf2-961d02a4c3fe None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:06.361480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180efa9-d80c-4cf4-b065-9706903f666c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:06.416968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10518673-d1fd-47af-9eca-0496a285cd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:06.416968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10518673-d1fd-47af-9eca-0496a285cd19 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:06.416968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10518673-d1fd-47af-9eca-0496a285cd19 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:06.416968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10518673-d1fd-47af-9eca-0496a285cd19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:06.430668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10518673-d1fd-47af-9eca-0496a285cd19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:06.432896 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10518673-d1fd-47af-9eca-0496a285cd19 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:06.454083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-10518673-d1fd-47af-9eca-0496a285cd19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:06.454083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:06.454083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:06.454083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:06.454083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:06.454083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:06.454083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:06.454083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:06.454083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:06.454083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:06.454083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:06.454083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:06.454083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:06.454083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:06.454083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:06.455038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:06.455952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:06.455952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:06.455952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:06.455952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:06.459963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 306/634] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:06.461381 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cfa9d-7c11-422b-a61d-7d98d53065d7 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:06.461674 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cfa9d-7c11-422b-a61d-7d98d53065d7 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:06.461956 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cfa9d-7c11-422b-a61d-7d98d53065d7 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:06.463312 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 307/635] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:06.463780 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:06.464049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cfa9d-7c11-422b-a61d-7d98d53065d7 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:06.473557 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b656d4-f714-45a3-bdf2-961d02a4c3fe None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:06.493754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:06.517523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc604a93-b760-4f41-9c60-bd8205b38261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:06.517523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc604a93-b760-4f41-9c60-bd8205b38261 None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:06.517523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc604a93-b760-4f41-9c60-bd8205b38261 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:06.517523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc604a93-b760-4f41-9c60-bd8205b38261 None admin] PATH_INFO: `/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/833a474f828f4349bea0d1649461c77e/roles/05f3681d0a644607934e281f75ecd675` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:06.534288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc604a93-b760-4f41-9c60-bd8205b38261 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc61fe15894891a9054911321f4ac3, user_id=833a474f828f4349bea0d1649461c77e, role_id=05f3681d0a644607934e281f75ecd675)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:06.535975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc604a93-b760-4f41-9c60-bd8205b38261 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:06.559752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 329/636] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:39:06 2022] PUT /identity/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/833a474f828f4349bea0d1649461c77e/roles/05f3681d0a644607934e281f75ecd675 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:06.559752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc604a93-b760-4f41-9c60-bd8205b38261 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:06.611683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cfa9d-7c11-422b-a61d-7d98d53065d7 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:06.663913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e2679-beb5-4834-9894-af470e7fae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:06.665390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e2679-beb5-4834-9894-af470e7fae98 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:06.666043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e2679-beb5-4834-9894-af470e7fae98 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:06.666444 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e2679-beb5-4834-9894-af470e7fae98 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:06.667308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e2679-beb5-4834-9894-af470e7fae98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:06.667990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e2679-beb5-4834-9894-af470e7fae98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:06.670032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e2679-beb5-4834-9894-af470e7fae98 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:06.688351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 308/637] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:06.688777 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc604a93-b760-4f41-9c60-bd8205b38261 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:06.689946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f0f2e-61a9-465d-b37b-c3ba335cfa2c None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:06.690307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f0f2e-61a9-465d-b37b-c3ba335cfa2c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:06.690971 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f0f2e-61a9-465d-b37b-c3ba335cfa2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:06.720855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-895f0f2e-61a9-465d-b37b-c3ba335cfa2c None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:06.722301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-895f0f2e-61a9-465d-b37b-c3ba335cfa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:06.732541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c7c78-0141-446c-ac21-536d1502bb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:06.732966 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c7c78-0141-446c-ac21-536d1502bb68 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:06.733249 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c7c78-0141-446c-ac21-536d1502bb68 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:06.733522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c7c78-0141-446c-ac21-536d1502bb68 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:06.734249 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c7c78-0141-446c-ac21-536d1502bb68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:06.734556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c7c78-0141-446c-ac21-536d1502bb68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:06.736012 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c7c78-0141-446c-ac21-536d1502bb68 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:06.753585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 330/638] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:06.753585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c7c78-0141-446c-ac21-536d1502bb68 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:06.763697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:06.820582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 309/639] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:39:06.821946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd2831-93b5-48b3-9c45-36c3668cf927 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:06.822231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd2831-93b5-48b3-9c45-36c3668cf927 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:06.822508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd2831-93b5-48b3-9c45-36c3668cf927 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:06.831880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 310/640] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:06 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:06.834075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd2831-93b5-48b3-9c45-36c3668cf927 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:06.849068 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:06.897672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c7c78-0141-446c-ac21-536d1502bb68 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:06.938102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d8564-6cd0-4b02-ae85-c22206450473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:06.939204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d8564-6cd0-4b02-ae85-c22206450473 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:06.939490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d8564-6cd0-4b02-ae85-c22206450473 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:06.939793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d8564-6cd0-4b02-ae85-c22206450473 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:06.947863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d8564-6cd0-4b02-ae85-c22206450473 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:06.949315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d8564-6cd0-4b02-ae85-c22206450473 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:06.955798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd2831-93b5-48b3-9c45-36c3668cf927 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:06.965042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-cd0d8564-6cd0-4b02-ae85-c22206450473 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:06.965042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:06.965042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:06.965042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:06.965042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:06.965042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:06.965042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:06.965042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:06.965042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:06.965042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:06.965042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:06.965042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:06.965042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:06.965042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:06.965042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:06.966663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:06.968389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:06.968389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:06.968389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:06.968389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:06.968389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:06.968389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:06.968389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:06.968389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:06.971043 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 331/641] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:06.972928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3189eed-3d11-4aa7-9dcc-9d2f71121fb8 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:06.973135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3189eed-3d11-4aa7-9dcc-9d2f71121fb8 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:06.973285 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3189eed-3d11-4aa7-9dcc-9d2f71121fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:06.993174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d3189eed-3d11-4aa7-9dcc-9d2f71121fb8 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:06.994501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d3189eed-3d11-4aa7-9dcc-9d2f71121fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:07.003698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e98f6-56bb-430e-97c6-86d3aff77441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:07.004551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e98f6-56bb-430e-97c6-86d3aff77441 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:07.004859 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e98f6-56bb-430e-97c6-86d3aff77441 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:07.005115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e98f6-56bb-430e-97c6-86d3aff77441 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:07.013524 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e98f6-56bb-430e-97c6-86d3aff77441 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:07.014999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e98f6-56bb-430e-97c6-86d3aff77441 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-8c1e98f6-56bb-430e-97c6-86d3aff77441 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:07.031144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:07.032242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:07.033181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:07.033181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:07.033181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:07.033181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:07.033181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:07.033181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:07.033181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:07.033181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:07.033181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:39:07.033181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:07.033181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 311/642] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:39:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:07.034277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e98f6-56bb-430e-97c6-86d3aff77441 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:07.068932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:07.080262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 332/643] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:39:07.080509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3189eed-3d11-4aa7-9dcc-9d2f71121fb8 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:07.126357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:07.167091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e98f6-56bb-430e-97c6-86d3aff77441 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:07.208569 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3189eed-3d11-4aa7-9dcc-9d2f71121fb8 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:07.211226 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedaf66-9762-44bb-9840-f316a8a8a713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:07.212066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedaf66-9762-44bb-9840-f316a8a8a713 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:07.212350 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedaf66-9762-44bb-9840-f316a8a8a713 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:07.212625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedaf66-9762-44bb-9840-f316a8a8a713 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:07.227054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedaf66-9762-44bb-9840-f316a8a8a713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-5eedaf66-9762-44bb-9840-f316a8a8a713 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:07.235670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:07.237094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:07.237972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:07.237972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:07.237972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:07.237972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:07.237972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:07.237972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:07.237972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:07.237972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:07.237972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:07.237972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:07.237972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:07.237972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:07.237972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 312/644] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:39:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:07.237972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedaf66-9762-44bb-9840-f316a8a8a713 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:07.252378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fc396-1851-46b6-b6e4-fbf327c21f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:07.253057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fc396-1851-46b6-b6e4-fbf327c21f4c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:07.253342 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fc396-1851-46b6-b6e4-fbf327c21f4c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:07.253621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fc396-1851-46b6-b6e4-fbf327c21f4c None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:07.254430 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fc396-1851-46b6-b6e4-fbf327c21f4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:07.254732 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fc396-1851-46b6-b6e4-fbf327c21f4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:07.256423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fc396-1851-46b6-b6e4-fbf327c21f4c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:07.268416 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 333/645] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:07.269265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fc396-1851-46b6-b6e4-fbf327c21f4c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:07.337312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:07.362747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedaf66-9762-44bb-9840-f316a8a8a713 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:07.386337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fc396-1851-46b6-b6e4-fbf327c21f4c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:07.395611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:07.410366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe78a47-d80b-4482-a181-a3c64b1d340d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:07.411056 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe78a47-d80b-4482-a181-a3c64b1d340d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:07.411345 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe78a47-d80b-4482-a181-a3c64b1d340d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:07.412247 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe78a47-d80b-4482-a181-a3c64b1d340d None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:07.413115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe78a47-d80b-4482-a181-a3c64b1d340d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:07.413424 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe78a47-d80b-4482-a181-a3c64b1d340d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:07.414909 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe78a47-d80b-4482-a181-a3c64b1d340d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:07.437416 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbf19b-8968-475d-8635-a1131210d880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:07.437747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 313/646] 10.222.0.60 () {56 vars in 1137 bytes} [Thu Jul 21 20:39:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:07.438582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe78a47-d80b-4482-a181-a3c64b1d340d None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:07.439057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbf19b-8968-475d-8635-a1131210d880 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:07.439451 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbf19b-8968-475d-8635-a1131210d880 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:07.439859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbf19b-8968-475d-8635-a1131210d880 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:07.454047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbf19b-8968-475d-8635-a1131210d880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-58bbf19b-8968-475d-8635-a1131210d880 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:07.462016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:07.463198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:07.464096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:07.464096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:07.464096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:07.464096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:07.464096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:07.464096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:07.464096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:07.464096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:07.464096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:07.464096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:07.464096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:07.464096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:07.464096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 334/647] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:39:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:07.464793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbf19b-8968-475d-8635-a1131210d880 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:07.562186 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe78a47-d80b-4482-a181-a3c64b1d340d None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:07.586115 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbf19b-8968-475d-8635-a1131210d880 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:07.604762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9631498-999b-4b01-b811-e4f29ea7c460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:07.605439 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9631498-999b-4b01-b811-e4f29ea7c460 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:07.605727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9631498-999b-4b01-b811-e4f29ea7c460 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:07.606002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9631498-999b-4b01-b811-e4f29ea7c460 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:07.606758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9631498-999b-4b01-b811-e4f29ea7c460 None admin] RBAC: Adding query filter params (name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:07.607064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9631498-999b-4b01-b811-e4f29ea7c460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:07.608476 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9631498-999b-4b01-b811-e4f29ea7c460 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:07.609384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:07.617788 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 314/648] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:39:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:07.620886 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9631498-999b-4b01-b811-e4f29ea7c460 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:07.630464 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6885f30-e860-4c9e-a25b-d10a0b313493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:07.631286 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6885f30-e860-4c9e-a25b-d10a0b313493 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:07.631574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6885f30-e860-4c9e-a25b-d10a0b313493 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:07.631913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6885f30-e860-4c9e-a25b-d10a0b313493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:07.632776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6885f30-e860-4c9e-a25b-d10a0b313493 None admin] RBAC: Adding query filter params (role.id=05f3681d0a644607934e281f75ecd675, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=77812b356c974d2b80102217956169db) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:07.633151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6885f30-e860-4c9e-a25b-d10a0b313493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:07.634672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6885f30-e860-4c9e-a25b-d10a0b313493 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:07.641320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 335/649] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:39:07 2022] GET /identity/v3/role_assignments?user.id=77812b356c974d2b80102217956169db&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=05f3681d0a644607934e281f75ecd675 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:07.652801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6885f30-e860-4c9e-a25b-d10a0b313493 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:07.699164 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:07.796942 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9631498-999b-4b01-b811-e4f29ea7c460 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:07.798628 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6885f30-e860-4c9e-a25b-d10a0b313493 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:07.841098 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08055d5c-0e20-4744-9b79-ddce6856a9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:07.842636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08055d5c-0e20-4744-9b79-ddce6856a9db None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:07.842924 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08055d5c-0e20-4744-9b79-ddce6856a9db None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:07.843204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08055d5c-0e20-4744-9b79-ddce6856a9db None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:07.843999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08055d5c-0e20-4744-9b79-ddce6856a9db None admin] RBAC: Adding query filter params (name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:07.844368 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08055d5c-0e20-4744-9b79-ddce6856a9db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:07.844887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdb40e-200f-46cb-8d49-60b33ad1023c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:07.845538 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdb40e-200f-46cb-8d49-60b33ad1023c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:07.845725 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08055d5c-0e20-4744-9b79-ddce6856a9db None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:07.845822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdb40e-200f-46cb-8d49-60b33ad1023c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:07.846074 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdb40e-200f-46cb-8d49-60b33ad1023c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:07.856401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 336/650] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:39:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:07.857168 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08055d5c-0e20-4744-9b79-ddce6856a9db None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:07.920108 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:07.949703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdb40e-200f-46cb-8d49-60b33ad1023c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:07.951073 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdb40e-200f-46cb-8d49-60b33ad1023c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.004321 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:08.004775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08055d5c-0e20-4744-9b79-ddce6856a9db None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.054481 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da8d7b-d874-47a8-9029-56d001630f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.055576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da8d7b-d874-47a8-9029-56d001630f7d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.056030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da8d7b-d874-47a8-9029-56d001630f7d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.056480 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da8d7b-d874-47a8-9029-56d001630f7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.070022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da8d7b-d874-47a8-9029-56d001630f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.071898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da8d7b-d874-47a8-9029-56d001630f7d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.079864 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 315/651] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:39:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:39:08.081191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdb40e-200f-46cb-8d49-60b33ad1023c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.094902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-12da8d7b-d874-47a8-9029-56d001630f7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:08.094902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:08.094902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:08.094902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:08.094902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:08.094902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:08.094902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:08.094902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:08.094902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:08.094902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:08.094902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:08.094902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:08.094902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:08.094902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:08.094902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:08.096091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:08.097026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:08.097026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:08.097026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:08.097026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:08.099182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 337/652] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:08.104757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da8d7b-d874-47a8-9029-56d001630f7d None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.237612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdb40e-200f-46cb-8d49-60b33ad1023c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.273497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:08.284634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da8d7b-d874-47a8-9029-56d001630f7d None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.306669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b445d08-f03a-4de0-b0e3-7fcec9d1b808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.313268 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b445d08-f03a-4de0-b0e3-7fcec9d1b808 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.313575 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b445d08-f03a-4de0-b0e3-7fcec9d1b808 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.316529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b445d08-f03a-4de0-b0e3-7fcec9d1b808 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.345159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b445d08-f03a-4de0-b0e3-7fcec9d1b808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.349128 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b445d08-f03a-4de0-b0e3-7fcec9d1b808 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.371862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-0b445d08-f03a-4de0-b0e3-7fcec9d1b808 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:08.371862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:08.371862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:08.371862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:08.371862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:08.371862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:08.371862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:08.371862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:08.371862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:08.371862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:08.371862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:08.371862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:08.371862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:08.371862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:08.371862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:08.373217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:08.374092 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:08.374092 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:08.374092 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:08.374092 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:08.375052 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:08.383789 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 316/653] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:08.384338 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a912a-bc20-4981-b7e3-f37dd73d5e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.384502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b445d08-f03a-4de0-b0e3-7fcec9d1b808 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.384763 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a912a-bc20-4981-b7e3-f37dd73d5e30 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.384763 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a912a-bc20-4981-b7e3-f37dd73d5e30 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.384763 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a912a-bc20-4981-b7e3-f37dd73d5e30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.406586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a912a-bc20-4981-b7e3-f37dd73d5e30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.411214 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a912a-bc20-4981-b7e3-f37dd73d5e30 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.461669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-e64a912a-bc20-4981-b7e3-f37dd73d5e30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:08.461669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:08.461669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:08.461669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:08.461669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:08.461669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:08.461669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:08.461669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:08.461669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:08.461669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:08.461669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:08.461669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:08.461669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:08.461669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:08.461669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:08.462627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:08.463515 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:08.463515 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:08.463515 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:08.463515 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:08.466576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 338/654] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:08.469783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a912a-bc20-4981-b7e3-f37dd73d5e30 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.510229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b445d08-f03a-4de0-b0e3-7fcec9d1b808 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.553988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71934a4b-2613-417a-8b2f-ab77ab65e3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.555821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71934a4b-2613-417a-8b2f-ab77ab65e3dd None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.556110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71934a4b-2613-417a-8b2f-ab77ab65e3dd None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.556392 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71934a4b-2613-417a-8b2f-ab77ab65e3dd None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.557147 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71934a4b-2613-417a-8b2f-ab77ab65e3dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:08.557457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71934a4b-2613-417a-8b2f-ab77ab65e3dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.559553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71934a4b-2613-417a-8b2f-ab77ab65e3dd None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.577368 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 317/655] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:08.578619 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71934a4b-2613-417a-8b2f-ab77ab65e3dd None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.649580 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:08.706031 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a912a-bc20-4981-b7e3-f37dd73d5e30 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.715953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71934a4b-2613-417a-8b2f-ab77ab65e3dd None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.776705 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c936a2-6c81-4b98-b062-59687cb08f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.777378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c936a2-6c81-4b98-b062-59687cb08f31 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.777716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c936a2-6c81-4b98-b062-59687cb08f31 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.777999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c936a2-6c81-4b98-b062-59687cb08f31 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.778792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c936a2-6c81-4b98-b062-59687cb08f31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:08.779104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c936a2-6c81-4b98-b062-59687cb08f31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.780561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c936a2-6c81-4b98-b062-59687cb08f31 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.789732 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bab552-1de1-45bd-8694-0496ebc348c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.790415 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bab552-1de1-45bd-8694-0496ebc348c6 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.790701 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bab552-1de1-45bd-8694-0496ebc348c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.790980 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bab552-1de1-45bd-8694-0496ebc348c6 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.791742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bab552-1de1-45bd-8694-0496ebc348c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:08.792066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bab552-1de1-45bd-8694-0496ebc348c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.793458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bab552-1de1-45bd-8694-0496ebc348c6 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.802021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 339/656] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:08.802021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c936a2-6c81-4b98-b062-59687cb08f31 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.811810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 318/657] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:08.811810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dfa3d-61b0-4790-8a83-a6c4df97cb83 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.811810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dfa3d-61b0-4790-8a83-a6c4df97cb83 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.811810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dfa3d-61b0-4790-8a83-a6c4df97cb83 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.811810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 319/658] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:08.811810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284dfa3d-61b0-4790-8a83-a6c4df97cb83 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:08.813386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:08.920598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c936a2-6c81-4b98-b062-59687cb08f31 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.929340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284dfa3d-61b0-4790-8a83-a6c4df97cb83 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:08.952026 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:08.965795 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a50f9-8d00-4553-8843-db0eb8ed494d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.966625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a50f9-8d00-4553-8843-db0eb8ed494d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.966918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a50f9-8d00-4553-8843-db0eb8ed494d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.967201 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a50f9-8d00-4553-8843-db0eb8ed494d None admin] PATH_INFO: `/v3/users/glance` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.968136 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cec55d-6d87-44cb-8d1d-1f99fb449468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:08.969142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cec55d-6d87-44cb-8d1d-1f99fb449468 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:08.969666 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cec55d-6d87-44cb-8d1d-1f99fb449468 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:08.970096 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cec55d-6d87-44cb-8d1d-1f99fb449468 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:08.979695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a50f9-8d00-4553-8843-db0eb8ed494d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.981221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a50f9-8d00-4553-8843-db0eb8ed494d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.981464 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cec55d-6d87-44cb-8d1d-1f99fb449468 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:08.983400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cec55d-6d87-44cb-8d1d-1f99fb449468 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-fb2a50f9-8d00-4553-8843-db0eb8ed494d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:08.999973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:09.001009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:09.007423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:09.007423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:09.007423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:09.007423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:09.007423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:09.007423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:09.007423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:09.007423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:09.007423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 20:39:09.007423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:09.007423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 340/659] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:39:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-a6cec55d-6d87-44cb-8d1d-1f99fb449468 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:09.008057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:09.008898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:09.011452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:09.011452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:09.011452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:09.011452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:09.011452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:09.011452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:09.011452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:09.011452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:09.011452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 20:39:09.011452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:09.012841 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a50f9-8d00-4553-8843-db0eb8ed494d None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.012999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 320/660] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:39:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:09.012999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36da75-48b8-44b0-a10b-43457928c2f5 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.012999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36da75-48b8-44b0-a10b-43457928c2f5 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.012999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36da75-48b8-44b0-a10b-43457928c2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.032254 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-7f36da75-48b8-44b0-a10b-43457928c2f5 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:09.033485 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7f36da75-48b8-44b0-a10b-43457928c2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:09.088341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:09.123198 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 321/661] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:39:09.124222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36da75-48b8-44b0-a10b-43457928c2f5 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.144693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a50f9-8d00-4553-8843-db0eb8ed494d None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.186392 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3ec21-1259-47bd-9817-13428f6ab663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.187185 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3ec21-1259-47bd-9817-13428f6ab663 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.187468 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3ec21-1259-47bd-9817-13428f6ab663 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.187774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3ec21-1259-47bd-9817-13428f6ab663 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.196997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3ec21-1259-47bd-9817-13428f6ab663 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.198446 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3ec21-1259-47bd-9817-13428f6ab663 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.218366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-1ab3ec21-1259-47bd-9817-13428f6ab663 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:09.218366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:09.218366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:09.218366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:09.218366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:09.218366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:09.218366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:09.218366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:09.218366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:09.218366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:09.218366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:09.218366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:09.218366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:09.218366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:09.218366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:09.219526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:09.220458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:09.220458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:09.220458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:09.220458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:09.220458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:09.220458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:09.220458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:09.220458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:09.220458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 341/662] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:09.220458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3ec21-1259-47bd-9817-13428f6ab663 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.245413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36da75-48b8-44b0-a10b-43457928c2f5 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.259259 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:09.288242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671b116-8665-4e52-87c4-f38de442aaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.288955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671b116-8665-4e52-87c4-f38de442aaff None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.289240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671b116-8665-4e52-87c4-f38de442aaff None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.289659 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671b116-8665-4e52-87c4-f38de442aaff None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.290504 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671b116-8665-4e52-87c4-f38de442aaff None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:09.290818 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671b116-8665-4e52-87c4-f38de442aaff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.292439 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671b116-8665-4e52-87c4-f38de442aaff None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.314722 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 322/663] 10.222.0.60 () {56 vars in 1137 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:09.316206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671b116-8665-4e52-87c4-f38de442aaff None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.339695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3ec21-1259-47bd-9817-13428f6ab663 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.367684 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:09.379365 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4e412-b7b5-4d4b-9827-c57af0850381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.380086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4e412-b7b5-4d4b-9827-c57af0850381 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.380365 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4e412-b7b5-4d4b-9827-c57af0850381 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.380639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4e412-b7b5-4d4b-9827-c57af0850381 None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.381486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4e412-b7b5-4d4b-9827-c57af0850381 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:09.382176 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4e412-b7b5-4d4b-9827-c57af0850381 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.383986 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4e412-b7b5-4d4b-9827-c57af0850381 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.404220 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 342/664] 10.222.0.60 () {56 vars in 1139 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:09.404220 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4e412-b7b5-4d4b-9827-c57af0850381 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.469675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671b116-8665-4e52-87c4-f38de442aaff None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.515095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a387c40-ddd7-4538-8a6d-22f562aa18d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.516088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a387c40-ddd7-4538-8a6d-22f562aa18d9 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.516376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a387c40-ddd7-4538-8a6d-22f562aa18d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.516979 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a387c40-ddd7-4538-8a6d-22f562aa18d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.525197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4e412-b7b5-4d4b-9827-c57af0850381 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.530306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a387c40-ddd7-4538-8a6d-22f562aa18d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-3a387c40-ddd7-4538-8a6d-22f562aa18d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:09.538819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:09.539866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:09.540689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:09.540689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:09.540689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:09.540689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:09.540689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:09.540689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:09.540689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:09.540689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:09.540689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:09.540689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:09.540689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:09.540689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:09.540689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 323/665] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:09.541377 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a387c40-ddd7-4538-8a6d-22f562aa18d9 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.571303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e5e46-7254-464a-9e9b-8041308f130a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.572003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e5e46-7254-464a-9e9b-8041308f130a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.572285 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e5e46-7254-464a-9e9b-8041308f130a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.572554 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e5e46-7254-464a-9e9b-8041308f130a None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.573342 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e5e46-7254-464a-9e9b-8041308f130a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:09.573648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e5e46-7254-464a-9e9b-8041308f130a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.575072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e5e46-7254-464a-9e9b-8041308f130a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.575512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:09.589331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 343/666] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:09.594766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e5e46-7254-464a-9e9b-8041308f130a None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.644519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:09.664827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a387c40-ddd7-4538-8a6d-22f562aa18d9 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.703639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a62327-ec2b-4588-b250-3ecb73997e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.704333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a62327-ec2b-4588-b250-3ecb73997e88 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.704609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a62327-ec2b-4588-b250-3ecb73997e88 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.704880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a62327-ec2b-4588-b250-3ecb73997e88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.705999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e5e46-7254-464a-9e9b-8041308f130a None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:09.743284 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e66de-1436-4478-a5eb-bcca2e715c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:09.744959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e66de-1436-4478-a5eb-bcca2e715c7a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:09.744959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e66de-1436-4478-a5eb-bcca2e715c7a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:09.744959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e66de-1436-4478-a5eb-bcca2e715c7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:09.784381 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a62327-ec2b-4588-b250-3ecb73997e88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.785952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a62327-ec2b-4588-b250-3ecb73997e88 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.817270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:09.834576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e66de-1436-4478-a5eb-bcca2e715c7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:09.836166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e66de-1436-4478-a5eb-bcca2e715c7a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:09.883113 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:09.909401 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 324/667] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:39:09.910258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a62327-ec2b-4588-b250-3ecb73997e88 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:09.960542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 344/668] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:39:09.961221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e66de-1436-4478-a5eb-bcca2e715c7a None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.026921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a62327-ec2b-4588-b250-3ecb73997e88 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.058326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:10.066135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e66de-1436-4478-a5eb-bcca2e715c7a None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.068423 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3461c71-c4a9-44d7-b25f-b941f3afa397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.069505 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3461c71-c4a9-44d7-b25f-b941f3afa397 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.069888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3461c71-c4a9-44d7-b25f-b941f3afa397 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.070160 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3461c71-c4a9-44d7-b25f-b941f3afa397 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.070923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3461c71-c4a9-44d7-b25f-b941f3afa397 None admin] RBAC: Adding query filter params (name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:10.071224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3461c71-c4a9-44d7-b25f-b941f3afa397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.072625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3461c71-c4a9-44d7-b25f-b941f3afa397 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.081125 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 325/669] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:10.081951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3461c71-c4a9-44d7-b25f-b941f3afa397 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.108083 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbaca0-0736-4c26-8eba-96204df0ff4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.108748 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbaca0-0736-4c26-8eba-96204df0ff4d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.109032 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbaca0-0736-4c26-8eba-96204df0ff4d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.109309 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbaca0-0736-4c26-8eba-96204df0ff4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.110151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbaca0-0736-4c26-8eba-96204df0ff4d None admin] RBAC: Adding query filter params (role.id=05f3681d0a644607934e281f75ecd675, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=a8eef7746fc4417e8d4de0b052f3eaa0) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:10.110463 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbaca0-0736-4c26-8eba-96204df0ff4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.111988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbaca0-0736-4c26-8eba-96204df0ff4d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.118694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 345/670] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:39:09 2022] GET /identity/v3/role_assignments?user.id=a8eef7746fc4417e8d4de0b052f3eaa0&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=05f3681d0a644607934e281f75ecd675 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:10.120181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca499a5-a11c-4f17-bb81-efbd7bc5c7e0 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.120472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca499a5-a11c-4f17-bb81-efbd7bc5c7e0 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.120752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca499a5-a11c-4f17-bb81-efbd7bc5c7e0 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.122183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 346/671] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:10.122840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca499a5-a11c-4f17-bb81-efbd7bc5c7e0 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.135805 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:10.222719 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3461c71-c4a9-44d7-b25f-b941f3afa397 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.283361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca499a5-a11c-4f17-bb81-efbd7bc5c7e0 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.289306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38512f6-7a31-4842-8127-4712f7033d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.290663 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38512f6-7a31-4842-8127-4712f7033d47 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.291925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38512f6-7a31-4842-8127-4712f7033d47 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.291925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38512f6-7a31-4842-8127-4712f7033d47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.310693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38512f6-7a31-4842-8127-4712f7033d47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.313499 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38512f6-7a31-4842-8127-4712f7033d47 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.342827 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b42b1a-2cfa-4593-a99e-d264dc05fec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.343622 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b42b1a-2cfa-4593-a99e-d264dc05fec2 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.343983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-e38512f6-7a31-4842-8127-4712f7033d47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:10.343983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:10.343983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:10.343983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:10.343983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:10.343983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:10.343983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:10.343983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:10.343983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:10.343983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:10.343983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:10.343983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:10.343983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:10.343983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:10.343983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:10.345587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:10.349409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:10.349409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:10.349409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:10.349409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:10.350241 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b42b1a-2cfa-4593-a99e-d264dc05fec2 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.350241 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b42b1a-2cfa-4593-a99e-d264dc05fec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.350680 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 326/672] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:10.350680 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38512f6-7a31-4842-8127-4712f7033d47 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.358681 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b42b1a-2cfa-4593-a99e-d264dc05fec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.361234 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b42b1a-2cfa-4593-a99e-d264dc05fec2 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.387623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-b1b42b1a-2cfa-4593-a99e-d264dc05fec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:10.387623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:10.387623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:10.387623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:10.387623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:10.387623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:10.387623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:10.387623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:10.387623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:10.387623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:10.387623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:10.387623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:10.387623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:10.387623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:10.387623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:10.389394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:10.390839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:10.390839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:10.390839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:10.390839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:10.390839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 347/673] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:10.390839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a41805-ca9a-4426-ae9c-53d164bb8ea4 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.390839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a41805-ca9a-4426-ae9c-53d164bb8ea4 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.390839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a41805-ca9a-4426-ae9c-53d164bb8ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.418489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-04a41805-ca9a-4426-ae9c-53d164bb8ea4 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:10.420436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-04a41805-ca9a-4426-ae9c-53d164bb8ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:10.428291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:10.500421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:10.502589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38512f6-7a31-4842-8127-4712f7033d47 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.519751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 348/674] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:39:10.519751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a41805-ca9a-4426-ae9c-53d164bb8ea4 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.547206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568f899-7b54-4e41-9ded-a7546fbd3719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.548625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568f899-7b54-4e41-9ded-a7546fbd3719 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.549119 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568f899-7b54-4e41-9ded-a7546fbd3719 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.549339 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568f899-7b54-4e41-9ded-a7546fbd3719 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.566254 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568f899-7b54-4e41-9ded-a7546fbd3719 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.569174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568f899-7b54-4e41-9ded-a7546fbd3719 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.589503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-c568f899-7b54-4e41-9ded-a7546fbd3719 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:10.589503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:10.589503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:10.589503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:10.589503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:10.589503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:10.589503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:10.589503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:10.589503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:10.589503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:10.589503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:10.589503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:10.589503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:10.589503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:10.589503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:10.590958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:10.592700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:10.592700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:10.592700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:10.592700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:10.592700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 327/675] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:10.592700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568f899-7b54-4e41-9ded-a7546fbd3719 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.641581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a41805-ca9a-4426-ae9c-53d164bb8ea4 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.680794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d3d6e-af1c-40e0-9b82-a83c6b78da8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.681483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d3d6e-af1c-40e0-9b82-a83c6b78da8d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.681762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d3d6e-af1c-40e0-9b82-a83c6b78da8d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.682043 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d3d6e-af1c-40e0-9b82-a83c6b78da8d None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.682783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d3d6e-af1c-40e0-9b82-a83c6b78da8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:10.683090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d3d6e-af1c-40e0-9b82-a83c6b78da8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.684523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d3d6e-af1c-40e0-9b82-a83c6b78da8d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.703831 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 349/676] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:10.703831 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d3d6e-af1c-40e0-9b82-a83c6b78da8d None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.711756 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:10.719326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568f899-7b54-4e41-9ded-a7546fbd3719 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.757498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668893e-578f-4532-a218-312cd4a7aca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.758203 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668893e-578f-4532-a218-312cd4a7aca5 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.758480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668893e-578f-4532-a218-312cd4a7aca5 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.758754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668893e-578f-4532-a218-312cd4a7aca5 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.759651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d668893e-578f-4532-a218-312cd4a7aca5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:10.759986 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d668893e-578f-4532-a218-312cd4a7aca5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.761432 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d668893e-578f-4532-a218-312cd4a7aca5 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:10.763208 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:10.778831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 328/677] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:10.778831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668893e-578f-4532-a218-312cd4a7aca5 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.847080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d3d6e-af1c-40e0-9b82-a83c6b78da8d None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.907609 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179a931-913c-42a7-8b77-f139e30c300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.908365 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179a931-913c-42a7-8b77-f139e30c300a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.908681 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179a931-913c-42a7-8b77-f139e30c300a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.908929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179a931-913c-42a7-8b77-f139e30c300a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.924940 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668893e-578f-4532-a218-312cd4a7aca5 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:10.929167 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179a931-913c-42a7-8b77-f139e30c300a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-3179a931-913c-42a7-8b77-f139e30c300a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:10.939370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:10.940465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:10.941295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:10.941295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:10.941295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:10.941295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:10.941295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:10.941295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:10.941295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:10.941295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:10.941295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:10.941295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:10.941295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:39:10.941295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:10.941941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 350/678] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:10.942824 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179a931-913c-42a7-8b77-f139e30c300a None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:10.981558 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5b689-8714-465f-b2ea-f52911105d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:10.983021 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5b689-8714-465f-b2ea-f52911105d6d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:10.983476 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5b689-8714-465f-b2ea-f52911105d6d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:10.983974 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5b689-8714-465f-b2ea-f52911105d6d None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:10.985270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5b689-8714-465f-b2ea-f52911105d6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:10.985777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5b689-8714-465f-b2ea-f52911105d6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:10.988268 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5b689-8714-465f-b2ea-f52911105d6d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.009540 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 329/679] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:11.009540 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5b689-8714-465f-b2ea-f52911105d6d None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.031405 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:11.076906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179a931-913c-42a7-8b77-f139e30c300a None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.089553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:11.115928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d09917-ef7f-4efd-ad8e-0dba2d7d3100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.116728 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d09917-ef7f-4efd-ad8e-0dba2d7d3100 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.117010 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d09917-ef7f-4efd-ad8e-0dba2d7d3100 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.117325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d09917-ef7f-4efd-ad8e-0dba2d7d3100 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.124566 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d09917-ef7f-4efd-ad8e-0dba2d7d3100 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.126028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d09917-ef7f-4efd-ad8e-0dba2d7d3100 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.144611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5b689-8714-465f-b2ea-f52911105d6d None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.151421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-79d09917-ef7f-4efd-ad8e-0dba2d7d3100 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:11.151421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:11.151421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:11.151421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:11.151421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:11.151421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:11.151421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:11.151421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:11.151421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:11.151421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:11.151421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:11.151421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:11.151421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:11.151421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:11.151421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:11.152511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:11.153433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:11.153433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:11.153433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:11.153433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:11.153433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:11.153433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:11.153433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:11.153433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:11.157387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 351/680] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:11.158317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d09917-ef7f-4efd-ad8e-0dba2d7d3100 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.204657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13440b5-4d82-45fa-ac5c-0de37cec334f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.205939 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13440b5-4d82-45fa-ac5c-0de37cec334f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.206400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13440b5-4d82-45fa-ac5c-0de37cec334f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.207057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13440b5-4d82-45fa-ac5c-0de37cec334f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.220301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13440b5-4d82-45fa-ac5c-0de37cec334f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.221772 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13440b5-4d82-45fa-ac5c-0de37cec334f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.238792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-e13440b5-4d82-45fa-ac5c-0de37cec334f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:11.238792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:11.238792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:11.238792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:11.238792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:11.238792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:11.238792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:11.238792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:11.238792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:11.238792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:11.238792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:11.238792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:11.238792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:11.238792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:11.238792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:11.240185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:11.241474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:11.241474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:11.241474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:11.241474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:11.241474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:11.241474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:11.241474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:11.241474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:11.241474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 330/681] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:11.241977 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13440b5-4d82-45fa-ac5c-0de37cec334f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.314934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d09917-ef7f-4efd-ad8e-0dba2d7d3100 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.335311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:11.376712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13440b5-4d82-45fa-ac5c-0de37cec334f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.378276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fcd26-4c60-458c-b2ad-bde0cd719ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.379049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fcd26-4c60-458c-b2ad-bde0cd719ffb None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.379406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fcd26-4c60-458c-b2ad-bde0cd719ffb None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.379797 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fcd26-4c60-458c-b2ad-bde0cd719ffb None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.380724 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62fcd26-4c60-458c-b2ad-bde0cd719ffb None admin] RBAC: Adding query filter params (name=admin) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:11.381333 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62fcd26-4c60-458c-b2ad-bde0cd719ffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.382823 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62fcd26-4c60-458c-b2ad-bde0cd719ffb None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.399614 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 352/682] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:39:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:11.399614 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fcd26-4c60-458c-b2ad-bde0cd719ffb None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.431234 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc512b-e04b-4c17-9fb5-3647d3ebbea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.431982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc512b-e04b-4c17-9fb5-3647d3ebbea8 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.432267 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc512b-e04b-4c17-9fb5-3647d3ebbea8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.432545 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc512b-e04b-4c17-9fb5-3647d3ebbea8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.441938 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc512b-e04b-4c17-9fb5-3647d3ebbea8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.444005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc512b-e04b-4c17-9fb5-3647d3ebbea8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.445324 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-89fc512b-e04b-4c17-9fb5-3647d3ebbea8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:11.463511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:11.464752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:11.465799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:11.465799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:11.465799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:11.465799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:11.465799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:11.465799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:11.465799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:11.465799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:11.465799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:39:11.465799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:11.465799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 331/683] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:39:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:11.465799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc512b-e04b-4c17-9fb5-3647d3ebbea8 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.545042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fcd26-4c60-458c-b2ad-bde0cd719ffb None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.589917 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb39ccf-8c0c-452e-aced-fd542831a661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.590595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb39ccf-8c0c-452e-aced-fd542831a661 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.590879 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb39ccf-8c0c-452e-aced-fd542831a661 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.591263 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb39ccf-8c0c-452e-aced-fd542831a661 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.592243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb39ccf-8c0c-452e-aced-fd542831a661 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:11.592552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb39ccf-8c0c-452e-aced-fd542831a661 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.594025 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb39ccf-8c0c-452e-aced-fd542831a661 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.607360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc512b-e04b-4c17-9fb5-3647d3ebbea8 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.613588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 353/684] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:11.613588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb39ccf-8c0c-452e-aced-fd542831a661 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.655029 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9befd46-3436-48f9-8f1a-22c77e7f4977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.656008 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9befd46-3436-48f9-8f1a-22c77e7f4977 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.656478 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9befd46-3436-48f9-8f1a-22c77e7f4977 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.656915 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9befd46-3436-48f9-8f1a-22c77e7f4977 None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.657884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9befd46-3436-48f9-8f1a-22c77e7f4977 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:11.658458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9befd46-3436-48f9-8f1a-22c77e7f4977 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.661117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9befd46-3436-48f9-8f1a-22c77e7f4977 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.664916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:11.680326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 332/685] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:11.681192 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9befd46-3436-48f9-8f1a-22c77e7f4977 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.734327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:11.748757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb39ccf-8c0c-452e-aced-fd542831a661 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.799970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d1264-24f7-4d45-b8bc-3052fc7417fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.801142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d1264-24f7-4d45-b8bc-3052fc7417fa None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.801625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d1264-24f7-4d45-b8bc-3052fc7417fa None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.802836 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d1264-24f7-4d45-b8bc-3052fc7417fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.816558 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d1264-24f7-4d45-b8bc-3052fc7417fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.818071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d1264-24f7-4d45-b8bc-3052fc7417fa None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.820748 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9befd46-3436-48f9-8f1a-22c77e7f4977 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:11.839561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-351d1264-24f7-4d45-b8bc-3052fc7417fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:11.839561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:11.839561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:11.839561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:11.839561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:11.839561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:11.839561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:11.839561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:11.839561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:11.839561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:11.839561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:11.839561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:11.839561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:11.839561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:11.839561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:11.840528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:11.841395 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:11.841395 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:11.841395 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:11.841395 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:11.845547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 354/686] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:11.846355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d1264-24f7-4d45-b8bc-3052fc7417fa None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.859854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fe02b-1c6a-4ec1-89d0-9d873027463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:11.860544 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fe02b-1c6a-4ec1-89d0-9d873027463d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:11.860826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fe02b-1c6a-4ec1-89d0-9d873027463d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:11.861101 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fe02b-1c6a-4ec1-89d0-9d873027463d None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:11.862992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9fe02b-1c6a-4ec1-89d0-9d873027463d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:11.863530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9fe02b-1c6a-4ec1-89d0-9d873027463d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:11.866690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9fe02b-1c6a-4ec1-89d0-9d873027463d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:11.886469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 333/687] 10.222.0.60 () {56 vars in 1135 bytes} [Thu Jul 21 20:39:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:11.887306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fe02b-1c6a-4ec1-89d0-9d873027463d None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:11.950540 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:11.992837 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d1264-24f7-4d45-b8bc-3052fc7417fa None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:12.008802 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fe02b-1c6a-4ec1-89d0-9d873027463d None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:12.037864 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370e70c-6030-4482-9e0a-4c4b605f3c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:12.038528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370e70c-6030-4482-9e0a-4c4b605f3c47 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.038808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370e70c-6030-4482-9e0a-4c4b605f3c47 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.039080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370e70c-6030-4482-9e0a-4c4b605f3c47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.039945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370e70c-6030-4482-9e0a-4c4b605f3c47 None admin] RBAC: Adding query filter params (role.id=05f3681d0a644607934e281f75ecd675, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=1d596de177e24ca5a254bda6644a07da) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:12.040254 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370e70c-6030-4482-9e0a-4c4b605f3c47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:12.041694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370e70c-6030-4482-9e0a-4c4b605f3c47 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:12.048661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 355/688] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:39:11 2022] GET /identity/v3/role_assignments?user.id=1d596de177e24ca5a254bda6644a07da&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=05f3681d0a644607934e281f75ecd675 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:12.049457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370e70c-6030-4482-9e0a-4c4b605f3c47 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:12.062881 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af95a35-c48d-41e4-869f-762bff224b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:12.063566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af95a35-c48d-41e4-869f-762bff224b2c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.063882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af95a35-c48d-41e4-869f-762bff224b2c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.064159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af95a35-c48d-41e4-869f-762bff224b2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.065025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af95a35-c48d-41e4-869f-762bff224b2c None admin] RBAC: Adding query filter params (role.id=05f3681d0a644607934e281f75ecd675, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=33aff48933844401b86eb808937e35c4) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:12.065344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af95a35-c48d-41e4-869f-762bff224b2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:12.066811 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af95a35-c48d-41e4-869f-762bff224b2c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:12.075385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:12.076864 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 334/689] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:39:11 2022] GET /identity/v3/role_assignments?user.id=33aff48933844401b86eb808937e35c4&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=05f3681d0a644607934e281f75ecd675 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:12.085541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af95a35-c48d-41e4-869f-762bff224b2c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:12.276486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370e70c-6030-4482-9e0a-4c4b605f3c47 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:12.295126 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af95a35-c48d-41e4-869f-762bff224b2c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:12.322241 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c7852-cf82-4304-8193-a5aeffc2e9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:12.322927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c7852-cf82-4304-8193-a5aeffc2e9c7 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.323140 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:12.323244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c7852-cf82-4304-8193-a5aeffc2e9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.323471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c7852-cf82-4304-8193-a5aeffc2e9c7 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.324262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c7852-cf82-4304-8193-a5aeffc2e9c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:12.325192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c7852-cf82-4304-8193-a5aeffc2e9c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:12.325970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c7852-cf82-4304-8193-a5aeffc2e9c7 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:12.342742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 356/690] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:12.349215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c7852-cf82-4304-8193-a5aeffc2e9c7 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:12.377182 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721a843-815b-45a6-a6bb-430903ea5556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:12.378262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721a843-815b-45a6-a6bb-430903ea5556 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.378749 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721a843-815b-45a6-a6bb-430903ea5556 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.379222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721a843-815b-45a6-a6bb-430903ea5556 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.450116 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:12.475290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721a843-815b-45a6-a6bb-430903ea5556 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:12.476910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721a843-815b-45a6-a6bb-430903ea5556 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:12.495666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c7852-cf82-4304-8193-a5aeffc2e9c7 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:12.543473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14ce72-efe6-45ac-8bd0-af304dbea4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:12.544377 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14ce72-efe6-45ac-8bd0-af304dbea4a9 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.544773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14ce72-efe6-45ac-8bd0-af304dbea4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.545059 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14ce72-efe6-45ac-8bd0-af304dbea4a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.553874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14ce72-efe6-45ac-8bd0-af304dbea4a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:12.555543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14ce72-efe6-45ac-8bd0-af304dbea4a9 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-4c14ce72-efe6-45ac-8bd0-af304dbea4a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:12.572770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:12.573984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:12.575006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:12.575006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:12.575006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:12.575006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:12.575006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:12.575006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:12.575006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:12.575006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:12.575006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:39:12.575006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:12.575006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 357/691] 10.222.0.60 () {56 vars in 1093 bytes} [Thu Jul 21 20:39:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:12.576123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d19af-0fb5-49bd-bb64-72e32a2311a1 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.576123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d19af-0fb5-49bd-bb64-72e32a2311a1 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.576356 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d19af-0fb5-49bd-bb64-72e32a2311a1 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.581572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 358/692] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:12.581572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d19af-0fb5-49bd-bb64-72e32a2311a1 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:12.611901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 335/693] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:39:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:39:12.613566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68d045-be1e-418c-a619-ba2d5bb467f2 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.613865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68d045-be1e-418c-a619-ba2d5bb467f2 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.614144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68d045-be1e-418c-a619-ba2d5bb467f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.633057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-5a68d045-be1e-418c-a619-ba2d5bb467f2 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:12.634282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5a68d045-be1e-418c-a619-ba2d5bb467f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:12.649804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:12.701130 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d19af-0fb5-49bd-bb64-72e32a2311a1 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:12.718468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 336/694] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:39:12.719745 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68d045-be1e-418c-a619-ba2d5bb467f2 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:12.735087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:12.745009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71055dba-ffa3-4fa1-86c7-37809698a9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:12.745847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71055dba-ffa3-4fa1-86c7-37809698a9be None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.746712 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71055dba-ffa3-4fa1-86c7-37809698a9be None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.746712 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71055dba-ffa3-4fa1-86c7-37809698a9be None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.747678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71055dba-ffa3-4fa1-86c7-37809698a9be None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:12.748127 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71055dba-ffa3-4fa1-86c7-37809698a9be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:12.749789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71055dba-ffa3-4fa1-86c7-37809698a9be None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:12.778155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 359/695] 10.222.0.60 () {56 vars in 1133 bytes} [Thu Jul 21 20:39:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:12.779382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71055dba-ffa3-4fa1-86c7-37809698a9be None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:12.837918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68d045-be1e-418c-a619-ba2d5bb467f2 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:12.876566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f4050-0644-40f6-bd20-4b47e6d73a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:12.877384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f4050-0644-40f6-bd20-4b47e6d73a74 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.877665 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f4050-0644-40f6-bd20-4b47e6d73a74 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.877939 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f4050-0644-40f6-bd20-4b47e6d73a74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.890511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f4050-0644-40f6-bd20-4b47e6d73a74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:12.892148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f4050-0644-40f6-bd20-4b47e6d73a74 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:12.900967 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:12.911656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71055dba-ffa3-4fa1-86c7-37809698a9be None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:12.913417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-c00f4050-0644-40f6-bd20-4b47e6d73a74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:12.913417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:12.913417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:12.913417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:12.913417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:12.913417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:12.913417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:12.913417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:12.913417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:12.913417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:12.913417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:12.913417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:12.913417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:12.913417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:12.913417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:12.914494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:12.915551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:12.915551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:12.915551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:12.915551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:12.922531 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 337/696] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:12.922531 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f4050-0644-40f6-bd20-4b47e6d73a74 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:12.967686 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511f37d-61eb-4f01-bb66-677c09aa8c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:12.969141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511f37d-61eb-4f01-bb66-677c09aa8c27 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:12.969449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511f37d-61eb-4f01-bb66-677c09aa8c27 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:12.969726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511f37d-61eb-4f01-bb66-677c09aa8c27 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:12.970394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511f37d-61eb-4f01-bb66-677c09aa8c27 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:12.972092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511f37d-61eb-4f01-bb66-677c09aa8c27 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-0511f37d-61eb-4f01-bb66-677c09aa8c27 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:12.992946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:12.994015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:12.994851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:12.994851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:12.994851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:12.994851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:12.994851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:12.994851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:12.994851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:12.994851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:12.994851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:12.994851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:12.994851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:12.994851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:12.994851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:12.994851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:39:12.994851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:12.999124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 360/697] 10.222.0.60 () {56 vars in 1129 bytes} [Thu Jul 21 20:39:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:13.000820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511f37d-61eb-4f01-bb66-677c09aa8c27 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:13.070622 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:13.070860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f4050-0644-40f6-bd20-4b47e6d73a74 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:13.117216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc795416-b04e-4c08-9dd5-2e5e228c4d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:13.117804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc795416-b04e-4c08-9dd5-2e5e228c4d7b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:13.118047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc795416-b04e-4c08-9dd5-2e5e228c4d7b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:13.118325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc795416-b04e-4c08-9dd5-2e5e228c4d7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:13.129468 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511f37d-61eb-4f01-bb66-677c09aa8c27 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:13.167547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2355d-792c-4ba1-a04e-688981beeddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:13.168644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2355d-792c-4ba1-a04e-688981beeddf None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:13.169132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2355d-792c-4ba1-a04e-688981beeddf None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:13.169607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2355d-792c-4ba1-a04e-688981beeddf None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:13.170555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2355d-792c-4ba1-a04e-688981beeddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:13.171082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2355d-792c-4ba1-a04e-688981beeddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:13.172709 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2355d-792c-4ba1-a04e-688981beeddf None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:13.191145 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 361/698] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:13.191601 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:13.194116 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2355d-792c-4ba1-a04e-688981beeddf None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:13.197632 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc795416-b04e-4c08-9dd5-2e5e228c4d7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:13.199442 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc795416-b04e-4c08-9dd5-2e5e228c4d7b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:13.308848 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2355d-792c-4ba1-a04e-688981beeddf None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:13.319762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 338/699] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:39:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 20:39:13.319762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc795416-b04e-4c08-9dd5-2e5e228c4d7b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:13.327639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:13.354986 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e7ea2-5b02-4a72-9204-f12c03a5a35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:13.356063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e7ea2-5b02-4a72-9204-f12c03a5a35d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:13.356629 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e7ea2-5b02-4a72-9204-f12c03a5a35d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:13.357134 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e7ea2-5b02-4a72-9204-f12c03a5a35d None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:13.358057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e7ea2-5b02-4a72-9204-f12c03a5a35d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:13.358580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e7ea2-5b02-4a72-9204-f12c03a5a35d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:13.360195 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e7ea2-5b02-4a72-9204-f12c03a5a35d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:13.366264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 362/700] 10.222.0.60 () {56 vars in 1124 bytes} [Thu Jul 21 20:39:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:13.367538 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e7ea2-5b02-4a72-9204-f12c03a5a35d None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:13.436487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:13.445452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc795416-b04e-4c08-9dd5-2e5e228c4d7b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:13.492364 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc698440-201c-420b-bbf4-77096c4d3f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:13.493606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc698440-201c-420b-bbf4-77096c4d3f12 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:13.493996 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e7ea2-5b02-4a72-9204-f12c03a5a35d None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:13.494679 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc698440-201c-420b-bbf4-77096c4d3f12 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:13.495201 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc698440-201c-420b-bbf4-77096c4d3f12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:13.504925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc698440-201c-420b-bbf4-77096c4d3f12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:13.507687 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc698440-201c-420b-bbf4-77096c4d3f12 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:13.526767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-dc698440-201c-420b-bbf4-77096c4d3f12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:13.526767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:13.526767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:13.526767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:13.526767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:13.526767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:13.526767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:13.526767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:13.526767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:13.526767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:13.526767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:13.526767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:13.526767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:13.526767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:13.526767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:13.527734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:13.528618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:13.528618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:13.528618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:13.528618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:13.528618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:13.528618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:13.528618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:13.528618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:13.530740 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 339/701] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:13.530740 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc698440-201c-420b-bbf4-77096c4d3f12 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:13.541078 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9e7bd-280c-44b1-be22-6392889da309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:13.541886 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9e7bd-280c-44b1-be22-6392889da309 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:13.542170 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9e7bd-280c-44b1-be22-6392889da309 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:13.542449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9e7bd-280c-44b1-be22-6392889da309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:13.555070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9e7bd-280c-44b1-be22-6392889da309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:13.556525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9e7bd-280c-44b1-be22-6392889da309 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:13.577717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-22d9e7bd-280c-44b1-be22-6392889da309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:13.577717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:13.577717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:13.577717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:13.577717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:13.577717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:13.577717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:13.577717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:13.577717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:13.577717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:13.577717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:13.577717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:13.577717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:13.577717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:13.577717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:13.579092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:13.581360 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:13.581360 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:13.581360 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:13.581360 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:13.584093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 363/702] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:13.585144 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9e7bd-280c-44b1-be22-6392889da309 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:13.617746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:13.663987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc698440-201c-420b-bbf4-77096c4d3f12 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:13.703649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9e7bd-280c-44b1-be22-6392889da309 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:13.704656 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5433a3c-d00d-4181-b953-1a4ae5b99b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:13.705587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5433a3c-d00d-4181-b953-1a4ae5b99b26 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:13.706090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5433a3c-d00d-4181-b953-1a4ae5b99b26 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:13.706569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5433a3c-d00d-4181-b953-1a4ae5b99b26 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:13.707504 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5433a3c-d00d-4181-b953-1a4ae5b99b26 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:13.708067 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5433a3c-d00d-4181-b953-1a4ae5b99b26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:13.709800 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5433a3c-d00d-4181-b953-1a4ae5b99b26 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:13.716552 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 340/703] 10.222.0.60 () {56 vars in 1124 bytes} [Thu Jul 21 20:39:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:13.717775 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5433a3c-d00d-4181-b953-1a4ae5b99b26 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:13.746360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:13.768746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b999e-d408-4adb-8f93-fc0fba67d47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:13.769459 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b999e-d408-4adb-8f93-fc0fba67d47d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:13.769759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b999e-d408-4adb-8f93-fc0fba67d47d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:13.770127 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b999e-d408-4adb-8f93-fc0fba67d47d None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:13.771634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b999e-d408-4adb-8f93-fc0fba67d47d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:13.771970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b999e-d408-4adb-8f93-fc0fba67d47d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:13.773877 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b999e-d408-4adb-8f93-fc0fba67d47d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:13.807773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 364/704] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:13.821362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b999e-d408-4adb-8f93-fc0fba67d47d None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:13.843845 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5433a3c-d00d-4181-b953-1a4ae5b99b26 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:13.887142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0e386-6aa7-4445-aaf9-035ef60f9faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:13.888302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0e386-6aa7-4445-aaf9-035ef60f9faf None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:13.888801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0e386-6aa7-4445-aaf9-035ef60f9faf None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:13.889283 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0e386-6aa7-4445-aaf9-035ef60f9faf None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:13.890243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0e386-6aa7-4445-aaf9-035ef60f9faf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:13.890777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0e386-6aa7-4445-aaf9-035ef60f9faf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:13.892458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0e386-6aa7-4445-aaf9-035ef60f9faf None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:13.907760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 341/705] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:13.916242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0e386-6aa7-4445-aaf9-035ef60f9faf None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:14.023675 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:14.041622 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b999e-d408-4adb-8f93-fc0fba67d47d None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:14.076150 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:14.092379 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6e552-fa36-4a59-aba2-aa2c3f3818ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:14.093431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6e552-fa36-4a59-aba2-aa2c3f3818ab None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:14.093989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6e552-fa36-4a59-aba2-aa2c3f3818ab None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:14.094271 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6e552-fa36-4a59-aba2-aa2c3f3818ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:14.095294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6e552-fa36-4a59-aba2-aa2c3f3818ab None admin] RBAC: Adding query filter params (role.id=05f3681d0a644607934e281f75ecd675, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=833a474f828f4349bea0d1649461c77e) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:14.095613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6e552-fa36-4a59-aba2-aa2c3f3818ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:14.097627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6e552-fa36-4a59-aba2-aa2c3f3818ab None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:14.099936 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0e386-6aa7-4445-aaf9-035ef60f9faf None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:14.106387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 365/706] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:39:13 2022] GET /identity/v3/role_assignments?user.id=833a474f828f4349bea0d1649461c77e&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=05f3681d0a644607934e281f75ecd675 => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:14.178603 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc0d26-ef44-4677-8c2d-e6ff686b80e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:14.179355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc0d26-ef44-4677-8c2d-e6ff686b80e0 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:14.179642 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc0d26-ef44-4677-8c2d-e6ff686b80e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:14.179960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc0d26-ef44-4677-8c2d-e6ff686b80e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:14.197053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc0d26-ef44-4677-8c2d-e6ff686b80e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:14.198583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc0d26-ef44-4677-8c2d-e6ff686b80e0 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:14.220155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-40bc0d26-ef44-4677-8c2d-e6ff686b80e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:14.220155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:14.220155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:14.220155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:14.220155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:14.220155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:14.220155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:14.220155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:14.220155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:14.220155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:14.220155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:14.220155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:14.220155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:14.220155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:14.220155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:14.221090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:14.221960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:14.221960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:14.221960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:14.221960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:14.221960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:14.221960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:14.221960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:14.221960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:14.222847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 342/707] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:14.228032 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6e552-fa36-4a59-aba2-aa2c3f3818ab None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:14.382497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6e552-fa36-4a59-aba2-aa2c3f3818ab None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:14.420777 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf31089-7b66-4361-8b3d-c769eccccf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:14.421452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf31089-7b66-4361-8b3d-c769eccccf90 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:14.421736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf31089-7b66-4361-8b3d-c769eccccf90 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:14.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf31089-7b66-4361-8b3d-c769eccccf90 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:14.422808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf31089-7b66-4361-8b3d-c769eccccf90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:14.423118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf31089-7b66-4361-8b3d-c769eccccf90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:14.424590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf31089-7b66-4361-8b3d-c769eccccf90 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:14.439140 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 366/708] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:14.445119 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc0d26-ef44-4677-8c2d-e6ff686b80e0 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:14.486576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e51f00-2864-44c2-81cb-40f845ea00ad None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:14.487165 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e51f00-2864-44c2-81cb-40f845ea00ad None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:14.487447 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e51f00-2864-44c2-81cb-40f845ea00ad None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:14.489147 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 367/709] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:14.495852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb85e3a-4df2-493a-8661-b2506b0fe075 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:14.496445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb85e3a-4df2-493a-8661-b2506b0fe075 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:14.496445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb85e3a-4df2-493a-8661-b2506b0fe075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:14.515424 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-0fb85e3a-4df2-493a-8661-b2506b0fe075 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:14.516681 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-0fb85e3a-4df2-493a-8661-b2506b0fe075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:14.520136 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:14.606307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc0d26-ef44-4677-8c2d-e6ff686b80e0 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:14.614987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 368/710] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:39:14.616676 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb4e8d-985c-48e0-86c3-c7e467904ff3 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:14.616975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb4e8d-985c-48e0-86c3-c7e467904ff3 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:14.617296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb4e8d-985c-48e0-86c3-c7e467904ff3 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:14.618940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 369/711] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:14.625652 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21845339-7b5a-463c-9295-41d681031052 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:14.626007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21845339-7b5a-463c-9295-41d681031052 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:14.626236 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21845339-7b5a-463c-9295-41d681031052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:14.647170 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-21845339-7b5a-463c-9295-41d681031052 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:14.648422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-21845339-7b5a-463c-9295-41d681031052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:14.658483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe5b45-3c43-4ac8-839b-173df27d6ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:14.659953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe5b45-3c43-4ac8-839b-173df27d6ab0 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:14.660477 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe5b45-3c43-4ac8-839b-173df27d6ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:14.661930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe5b45-3c43-4ac8-839b-173df27d6ab0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:14.663588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe5b45-3c43-4ac8-839b-173df27d6ab0 None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=77812b356c974d2b80102217956169db) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:14.664288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe5b45-3c43-4ac8-839b-173df27d6ab0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:14.667214 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe5b45-3c43-4ac8-839b-173df27d6ab0 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:14.675361 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 343/712] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:39:14 2022] GET /identity/v3/role_assignments?user.id=77812b356c974d2b80102217956169db&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:14.678186 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe5b45-3c43-4ac8-839b-173df27d6ab0 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:14.690761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:14.747779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 370/713] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:39:14.784503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21845339-7b5a-463c-9295-41d681031052 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:14.820411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe5b45-3c43-4ac8-839b-173df27d6ab0 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:14.867855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2c90c-14ff-4f04-9062-d4570b1e727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:14.868777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2c90c-14ff-4f04-9062-d4570b1e727c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:14.869129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2c90c-14ff-4f04-9062-d4570b1e727c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:14.869499 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2c90c-14ff-4f04-9062-d4570b1e727c None admin] PATH_INFO: `/v3/services/network` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:14.870235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2c90c-14ff-4f04-9062-d4570b1e727c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:14.871695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2c90c-14ff-4f04-9062-d4570b1e727c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-fee2c90c-14ff-4f04-9062-d4570b1e727c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:14.894153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:14.895207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:14.896411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:14.896411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:14.896411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:14.896411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:14.896411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:14.896411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:14.896411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:14.896411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:14.896411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:14.896411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:14.896411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:14.896411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:14.896411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:39:14.896411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:14.896411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 344/714] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:14 2022] GET /identity/v3/services/network => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:14.902060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2c90c-14ff-4f04-9062-d4570b1e727c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:14.938820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:14.987558 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:15.025711 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21845339-7b5a-463c-9295-41d681031052 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:15.033667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2c90c-14ff-4f04-9062-d4570b1e727c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:15.097616 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140ad46-39db-4f38-8b9e-5f2ea5796418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:15.098432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140ad46-39db-4f38-8b9e-5f2ea5796418 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:15.098711 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140ad46-39db-4f38-8b9e-5f2ea5796418 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:15.099976 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140ad46-39db-4f38-8b9e-5f2ea5796418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:15.119192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0140ad46-39db-4f38-8b9e-5f2ea5796418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:15.120410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36271377-5c2b-4dd7-bd66-ab3134fb2168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:15.121092 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36271377-5c2b-4dd7-bd66-ab3134fb2168 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:15.121371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36271377-5c2b-4dd7-bd66-ab3134fb2168 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:15.121641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36271377-5c2b-4dd7-bd66-ab3134fb2168 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:15.122360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36271377-5c2b-4dd7-bd66-ab3134fb2168 None admin] RBAC: Adding query filter params (name=network) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:15.122660 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36271377-5c2b-4dd7-bd66-ab3134fb2168 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:15.124087 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36271377-5c2b-4dd7-bd66-ab3134fb2168 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:15.126320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0140ad46-39db-4f38-8b9e-5f2ea5796418 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:15.137788 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 345/715] 10.222.0.60 () {56 vars in 1112 bytes} [Thu Jul 21 20:39:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:15.144506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36271377-5c2b-4dd7-bd66-ab3134fb2168 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:15.161216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-0140ad46-39db-4f38-8b9e-5f2ea5796418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:15.161216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:15.161216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:15.161216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:15.161216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:15.161216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:15.161216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:15.161216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:15.161216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:15.161216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:15.161216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:15.161216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:15.161216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:15.161216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:15.161216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:15.163187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:15.164120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:15.164120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:15.164120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:15.164120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:15.174279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 371/716] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:15.185349 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140ad46-39db-4f38-8b9e-5f2ea5796418 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:15.323393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140ad46-39db-4f38-8b9e-5f2ea5796418 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:15.362986 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acb58c-4f95-4de0-815b-9cfe8aec6661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:15.363665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acb58c-4f95-4de0-815b-9cfe8aec6661 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:15.364000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acb58c-4f95-4de0-815b-9cfe8aec6661 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:15.364281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acb58c-4f95-4de0-815b-9cfe8aec6661 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:15.365585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acb58c-4f95-4de0-815b-9cfe8aec6661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:15.365984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acb58c-4f95-4de0-815b-9cfe8aec6661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:15.371737 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acb58c-4f95-4de0-815b-9cfe8aec6661 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:15.380175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:15.380552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 372/717] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:15.392820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acb58c-4f95-4de0-815b-9cfe8aec6661 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:15.469721 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36271377-5c2b-4dd7-bd66-ab3134fb2168 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:15.513454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acb58c-4f95-4de0-815b-9cfe8aec6661 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:15.555432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ee528-6c1e-404f-81f4-2dd10299237b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:15.556619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ee528-6c1e-404f-81f4-2dd10299237b None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:15.556912 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ee528-6c1e-404f-81f4-2dd10299237b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:15.557192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ee528-6c1e-404f-81f4-2dd10299237b None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:15.557913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ee528-6c1e-404f-81f4-2dd10299237b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:15.559367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ee528-6c1e-404f-81f4-2dd10299237b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:15.584302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 373/718] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:39:15 2022] POST /identity/v3/users => generated 254 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:15.616753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e785c47-1c0b-4dfc-8b7c-5e8240ef4abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:15.617507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e785c47-1c0b-4dfc-8b7c-5e8240ef4abd None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:15.617839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e785c47-1c0b-4dfc-8b7c-5e8240ef4abd None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:15.618120 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e785c47-1c0b-4dfc-8b7c-5e8240ef4abd None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:15.618902 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e785c47-1c0b-4dfc-8b7c-5e8240ef4abd None admin] RBAC: Adding query filter params (type=network) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:15.619212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e785c47-1c0b-4dfc-8b7c-5e8240ef4abd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:15.620679 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e785c47-1c0b-4dfc-8b7c-5e8240ef4abd None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:15.622549 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:15.632077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 346/719] 10.222.0.60 () {56 vars in 1112 bytes} [Thu Jul 21 20:39:15 2022] GET /identity/v3/services?type=network => generated 124 bytes in 488 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:16.357365 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d8371-2f51-4556-9a62-80f6889e1b71 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:16.357772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d8371-2f51-4556-9a62-80f6889e1b71 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:16.357951 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d8371-2f51-4556-9a62-80f6889e1b71 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:16.359653 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 374/720] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:16.366024 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ae69b-bbd5-41fe-87f8-7d547354fe59 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:16.366642 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ae69b-bbd5-41fe-87f8-7d547354fe59 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:16.366642 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ae69b-bbd5-41fe-87f8-7d547354fe59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:16.387224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-092ae69b-bbd5-41fe-87f8-7d547354fe59 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:16.388476 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-092ae69b-bbd5-41fe-87f8-7d547354fe59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:16.482200 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 347/721] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:39:16.517019 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d8371-2f51-4556-9a62-80f6889e1b71 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:16.565385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:16.653849 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d8371-2f51-4556-9a62-80f6889e1b71 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:16.691087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd90c3d-cbc9-41b2-8559-b5de38a2aa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:16.691791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd90c3d-cbc9-41b2-8559-b5de38a2aa21 None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:16.692083 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd90c3d-cbc9-41b2-8559-b5de38a2aa21 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:16.692362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd90c3d-cbc9-41b2-8559-b5de38a2aa21 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:16.693375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd90c3d-cbc9-41b2-8559-b5de38a2aa21 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:16.694790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd90c3d-cbc9-41b2-8559-b5de38a2aa21 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:16.714455 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 375/722] 10.222.0.60 () {60 vars in 1141 bytes} [Thu Jul 21 20:39:16 2022] POST /identity/v3/services => generated 251 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:17.027261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70625a48-8259-4a66-919d-58d55f37609d None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:17.027685 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70625a48-8259-4a66-919d-58d55f37609d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:17.028097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70625a48-8259-4a66-919d-58d55f37609d None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:17.029880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 348/723] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:17.041986 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7ceaf-563f-41ac-892d-a215ab1d3407 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:17.042313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7ceaf-563f-41ac-892d-a215ab1d3407 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:17.042564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7ceaf-563f-41ac-892d-a215ab1d3407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:17.066800 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-07e7ceaf-563f-41ac-892d-a215ab1d3407 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:17.066993 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7f002-a4ed-442b-af34-2d1f8537401d None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:17.068136 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7f002-a4ed-442b-af34-2d1f8537401d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:17.068634 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7f002-a4ed-442b-af34-2d1f8537401d None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:17.069091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-07e7ceaf-563f-41ac-892d-a215ab1d3407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:17.070612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 349/724] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:17.077527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7908118-38b3-4019-828b-5a389655dc94 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:17.077794 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7908118-38b3-4019-828b-5a389655dc94 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:17.078076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7908118-38b3-4019-828b-5a389655dc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:17.097224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-f7908118-38b3-4019-828b-5a389655dc94 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:17.098426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f7908118-38b3-4019-828b-5a389655dc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:17.215207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 350/725] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:39:17.216932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 376/726] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 20:39:17.257121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7908118-38b3-4019-828b-5a389655dc94 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:17.263207 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7ceaf-563f-41ac-892d-a215ab1d3407 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:17.300210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:17.322117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:17.375104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7908118-38b3-4019-828b-5a389655dc94 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:17.400742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7ceaf-563f-41ac-892d-a215ab1d3407 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:17.415484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48f856-d42c-4150-816c-51bda2d6525b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:17.416321 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48f856-d42c-4150-816c-51bda2d6525b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:17.416869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48f856-d42c-4150-816c-51bda2d6525b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:17.417355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48f856-d42c-4150-816c-51bda2d6525b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:17.430178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef48f856-d42c-4150-816c-51bda2d6525b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:17.438911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8368f-42a9-406c-826e-e4ff3a880a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-ef48f856-d42c-4150-816c-51bda2d6525b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:17.439362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:17.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:17.441152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:17.441152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:17.441152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:17.441152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:17.441152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:17.441152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:17.441152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:17.441152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:17.441152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:17.441152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:17.441152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:39:17.441152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:17.441779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8368f-42a9-406c-826e-e4ff3a880a4d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:17.441779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8368f-42a9-406c-826e-e4ff3a880a4d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:17.441779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8368f-42a9-406c-826e-e4ff3a880a4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:17.449039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 351/727] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:39:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:17.450672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48f856-d42c-4150-816c-51bda2d6525b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:17.463747 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8368f-42a9-406c-826e-e4ff3a880a4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-c6e8368f-42a9-406c-826e-e4ff3a880a4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:17.469374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:17.470259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:17.471568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:17.471568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:17.471568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:17.471568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:17.471568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:17.471568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:17.471568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:17.471568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:17.471568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:17.471568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:17.471568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:39:17.471568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:17.471568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 377/728] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:39:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:17.480879 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8368f-42a9-406c-826e-e4ff3a880a4d None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:17.576830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48f856-d42c-4150-816c-51bda2d6525b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:17.596286 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:17.622182 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a44dd-1be2-4513-9b81-15055a2422d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:17.623218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a44dd-1be2-4513-9b81-15055a2422d2 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:17.623706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a44dd-1be2-4513-9b81-15055a2422d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:17.624222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a44dd-1be2-4513-9b81-15055a2422d2 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:17.625203 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a44dd-1be2-4513-9b81-15055a2422d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:17.625719 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a44dd-1be2-4513-9b81-15055a2422d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:17.627295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a44dd-1be2-4513-9b81-15055a2422d2 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:17.641921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 352/729] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:39:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:17.643579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a44dd-1be2-4513-9b81-15055a2422d2 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:17.650433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:17.670174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8368f-42a9-406c-826e-e4ff3a880a4d None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:17.717906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab75ca-51c7-4774-ac3a-4f297615d38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:17.718651 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab75ca-51c7-4774-ac3a-4f297615d38c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:17.718939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab75ca-51c7-4774-ac3a-4f297615d38c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:17.719216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab75ca-51c7-4774-ac3a-4f297615d38c None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:17.720015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab75ca-51c7-4774-ac3a-4f297615d38c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:17.720323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab75ca-51c7-4774-ac3a-4f297615d38c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:17.724924 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab75ca-51c7-4774-ac3a-4f297615d38c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:17.746509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 378/730] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:39:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:17.751856 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab75ca-51c7-4774-ac3a-4f297615d38c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:17.844744 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a44dd-1be2-4513-9b81-15055a2422d2 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:17.915763 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab75ca-51c7-4774-ac3a-4f297615d38c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:17.952471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f8316-0dfd-4ceb-ae11-9df7620b0de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:17.953292 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f8316-0dfd-4ceb-ae11-9df7620b0de4 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:17.953572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f8316-0dfd-4ceb-ae11-9df7620b0de4 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:17.953843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f8316-0dfd-4ceb-ae11-9df7620b0de4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:17.968257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd997c-5705-4e44-92c3-947e70f9ce83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:17.969069 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd997c-5705-4e44-92c3-947e70f9ce83 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:17.969347 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd997c-5705-4e44-92c3-947e70f9ce83 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:17.969616 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd997c-5705-4e44-92c3-947e70f9ce83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:17.981625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f8316-0dfd-4ceb-ae11-9df7620b0de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:17.983124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f8316-0dfd-4ceb-ae11-9df7620b0de4 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:17.999943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd997c-5705-4e44-92c3-947e70f9ce83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:18.001420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd997c-5705-4e44-92c3-947e70f9ce83 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:18.020893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:18.025801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-ad4f8316-0dfd-4ceb-ae11-9df7620b0de4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:18.025801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:18.025801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:18.025801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:18.025801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:18.025801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:18.025801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:18.025801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:18.025801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:18.025801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:18.025801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:18.025801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:18.025801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:18.025801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:18.025801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:18.026801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:18.027689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:18.027689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:18.027689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:18.027689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:18.039079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-59cd997c-5705-4e44-92c3-947e70f9ce83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:18.039079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:18.039079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:18.039079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:18.039079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:18.039079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:18.039079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:18.039079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:18.039079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:18.039079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:18.039079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:18.039079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:18.039079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:18.039079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:18.039079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:18.041398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:18.043573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:18.043573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:18.043573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:18.043573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:18.044124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 353/731] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:18.044267 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 379/732] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:18.048517 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f8316-0dfd-4ceb-ae11-9df7620b0de4 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:18.050257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd997c-5705-4e44-92c3-947e70f9ce83 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:18.108178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:18.205776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd997c-5705-4e44-92c3-947e70f9ce83 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:18.255019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f8316-0dfd-4ceb-ae11-9df7620b0de4 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:18.285042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af80348-9308-4e0b-8500-f9fdf9f92d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:18.285730 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af80348-9308-4e0b-8500-f9fdf9f92d4f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:18.286016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af80348-9308-4e0b-8500-f9fdf9f92d4f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:18.286351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af80348-9308-4e0b-8500-f9fdf9f92d4f None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:18.287113 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af80348-9308-4e0b-8500-f9fdf9f92d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:18.287419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af80348-9308-4e0b-8500-f9fdf9f92d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:18.288879 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af80348-9308-4e0b-8500-f9fdf9f92d4f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:18.301163 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c47cb-3316-4153-b57e-61cff9f61f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:18.302816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c47cb-3316-4153-b57e-61cff9f61f7f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:18.303105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c47cb-3316-4153-b57e-61cff9f61f7f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:18.303464 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c47cb-3316-4153-b57e-61cff9f61f7f None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:18.304343 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c47cb-3316-4153-b57e-61cff9f61f7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:18.304647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c47cb-3316-4153-b57e-61cff9f61f7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:18.306881 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c47cb-3316-4153-b57e-61cff9f61f7f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:18.315381 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 380/733] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:18.321560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af80348-9308-4e0b-8500-f9fdf9f92d4f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:18.326560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 354/734] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:18.332062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c47cb-3316-4153-b57e-61cff9f61f7f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:18.430618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:18.446609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:18.462541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c47cb-3316-4153-b57e-61cff9f61f7f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:18.479536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af80348-9308-4e0b-8500-f9fdf9f92d4f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:18.503176 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b0c6f-883b-4d1c-98b4-f5eb20dc8ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:18.504013 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b0c6f-883b-4d1c-98b4-f5eb20dc8ab8 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:18.504295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b0c6f-883b-4d1c-98b4-f5eb20dc8ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:18.504565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b0c6f-883b-4d1c-98b4-f5eb20dc8ab8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:18.513158 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b0c6f-883b-4d1c-98b4-f5eb20dc8ab8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:18.514616 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b0c6f-883b-4d1c-98b4-f5eb20dc8ab8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:18.518449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb7b4e-000d-42d2-a765-a8f53e38ef65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:18.519256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb7b4e-000d-42d2-a765-a8f53e38ef65 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:18.519543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb7b4e-000d-42d2-a765-a8f53e38ef65 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:18.519871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb7b4e-000d-42d2-a765-a8f53e38ef65 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:18.528609 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb7b4e-000d-42d2-a765-a8f53e38ef65 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:18.530203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb7b4e-000d-42d2-a765-a8f53e38ef65 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-9a3b0c6f-883b-4d1c-98b4-f5eb20dc8ab8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:18.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:18.532020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:18.532963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:18.532963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:18.532963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:18.532963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:18.532963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:18.532963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:18.532963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:18.532963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:18.532963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:39:18.532963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:18.532963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 355/735] 10.222.0.60 () {56 vars in 1093 bytes} [Thu Jul 21 20:39:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:18.534263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f171a6-1544-4eed-aa5d-fb5dbccf24cd None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:18.534263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f171a6-1544-4eed-aa5d-fb5dbccf24cd None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:18.534554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f171a6-1544-4eed-aa5d-fb5dbccf24cd None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:18.536088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 356/736] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:18.537386 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd029c-715f-4c8a-a5e8-adb5dadd0f0f None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:18.537670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd029c-715f-4c8a-a5e8-adb5dadd0f0f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:18.537939 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd029c-715f-4c8a-a5e8-adb5dadd0f0f None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:18.539306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 357/737] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:18.543988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd029c-715f-4c8a-a5e8-adb5dadd0f0f None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-8ebb7b4e-000d-42d2-a765-a8f53e38ef65 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:18.552281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:18.553417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:18.554435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:18.554435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:18.554435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:18.554435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:18.554435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:18.554435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:18.554435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:18.554435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:18.554435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:39:18.554435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:18.558542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 381/738] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:39:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:18.560293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396d6d7-0f7e-4dae-9cbb-64d74346e710 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:18.560758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396d6d7-0f7e-4dae-9cbb-64d74346e710 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:18.561041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396d6d7-0f7e-4dae-9cbb-64d74346e710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:18.579093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-6396d6d7-0f7e-4dae-9cbb-64d74346e710 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:18.580335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6396d6d7-0f7e-4dae-9cbb-64d74346e710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:18.663847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 382/739] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:39:18.663847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0c3f1-de11-49d8-885c-6922cae45acb None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:18.663847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0c3f1-de11-49d8-885c-6922cae45acb None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:18.663847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0c3f1-de11-49d8-885c-6922cae45acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:18.674282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd029c-715f-4c8a-a5e8-adb5dadd0f0f None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:18.688453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-23b0c3f1-de11-49d8-885c-6922cae45acb None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:18.689778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-23b0c3f1-de11-49d8-885c-6922cae45acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:18.712655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b7822-225d-4daa-b929-77bbd29cd6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:18.713332 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b7822-225d-4daa-b929-77bbd29cd6e6 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:18.713615 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b7822-225d-4daa-b929-77bbd29cd6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:18.713889 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b7822-225d-4daa-b929-77bbd29cd6e6 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:18.714726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b7822-225d-4daa-b929-77bbd29cd6e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:18.715029 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b7822-225d-4daa-b929-77bbd29cd6e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:18.716505 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b7822-225d-4daa-b929-77bbd29cd6e6 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:18.739449 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 358/740] 10.222.0.60 () {56 vars in 1133 bytes} [Thu Jul 21 20:39:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:18.739449 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b7822-225d-4daa-b929-77bbd29cd6e6 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:18.745892 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:18.750606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:18.773806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 383/741] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 20:39:18.774389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0c3f1-de11-49d8-885c-6922cae45acb None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:18.875081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b7822-225d-4daa-b929-77bbd29cd6e6 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:18.894679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0c3f1-de11-49d8-885c-6922cae45acb None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:18.923446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca68ae-2cbd-4c77-b0c8-46e97ffd05b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:18.924343 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca68ae-2cbd-4c77-b0c8-46e97ffd05b5 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:18.924955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca68ae-2cbd-4c77-b0c8-46e97ffd05b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:18.925234 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca68ae-2cbd-4c77-b0c8-46e97ffd05b5 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:18.926285 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca68ae-2cbd-4c77-b0c8-46e97ffd05b5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:18.926677 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca68ae-2cbd-4c77-b0c8-46e97ffd05b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:18.928909 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca68ae-2cbd-4c77-b0c8-46e97ffd05b5 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:18.934168 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ab6d6-674f-4f98-bb42-18d14341b4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:18.934973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ab6d6-674f-4f98-bb42-18d14341b4ee None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:18.935255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ab6d6-674f-4f98-bb42-18d14341b4ee None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:18.935530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ab6d6-674f-4f98-bb42-18d14341b4ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:18.948661 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 359/742] 10.222.0.60 () {56 vars in 1135 bytes} [Thu Jul 21 20:39:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:18.949354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca68ae-2cbd-4c77-b0c8-46e97ffd05b5 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:18.954210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ab6d6-674f-4f98-bb42-18d14341b4ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-2e9ab6d6-674f-4f98-bb42-18d14341b4ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:18.962590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:18.963752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:18.964690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:18.964690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:18.964690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:18.964690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:18.964690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:18.964690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:18.964690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:18.964690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:18.964690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:18.964690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:18.964690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:18.964690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:18.964690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 384/743] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:39:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:18.965423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ab6d6-674f-4f98-bb42-18d14341b4ee None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:19.013857 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:19.028209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:19.079427 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ab6d6-674f-4f98-bb42-18d14341b4ee None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:19.084222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca68ae-2cbd-4c77-b0c8-46e97ffd05b5 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:19.117256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c744c7-311e-4d24-aeaf-76afa57212bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:19.117921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c744c7-311e-4d24-aeaf-76afa57212bb None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:19.118203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c744c7-311e-4d24-aeaf-76afa57212bb None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:19.118476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c744c7-311e-4d24-aeaf-76afa57212bb None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:19.119201 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c744c7-311e-4d24-aeaf-76afa57212bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:19.120618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c744c7-311e-4d24-aeaf-76afa57212bb None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:19.124107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d38bf-76a9-4753-a8eb-129dc38973fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:19.125168 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d38bf-76a9-4753-a8eb-129dc38973fb None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:19.125456 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d38bf-76a9-4753-a8eb-129dc38973fb None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:19.125735 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d38bf-76a9-4753-a8eb-129dc38973fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:19.137253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d38bf-76a9-4753-a8eb-129dc38973fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:19.148123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 385/744] 10.222.0.60 () {60 vars in 1141 bytes} [Thu Jul 21 20:39:18 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:19.148359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d38bf-76a9-4753-a8eb-129dc38973fb None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:19.148456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c744c7-311e-4d24-aeaf-76afa57212bb None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:19.173335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-076d38bf-76a9-4753-a8eb-129dc38973fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:19.173335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:19.173335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:19.173335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:19.173335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:19.173335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:19.173335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:19.173335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:19.173335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:19.173335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:19.173335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:19.173335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:19.173335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:19.173335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:19.173335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:19.174369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:19.175270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:19.175270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:19.175270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:19.175270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:19.180269 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 360/745] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:19.181266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d38bf-76a9-4753-a8eb-129dc38973fb None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:19.286386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:19.302730 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c744c7-311e-4d24-aeaf-76afa57212bb None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:19.330265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d38bf-76a9-4753-a8eb-129dc38973fb None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:19.358028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:19.363353 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b271141-9930-44eb-9821-b9faeaf41e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:19.364264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b271141-9930-44eb-9821-b9faeaf41e1a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:19.364789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b271141-9930-44eb-9821-b9faeaf41e1a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:19.365294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b271141-9930-44eb-9821-b9faeaf41e1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:19.392747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab0092-259c-4c22-8a56-173428699737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:19.393436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab0092-259c-4c22-8a56-173428699737 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:19.393716 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab0092-259c-4c22-8a56-173428699737 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:19.393989 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab0092-259c-4c22-8a56-173428699737 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:19.395125 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab0092-259c-4c22-8a56-173428699737 None admin] RBAC: Adding query filter params (name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:19.395437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab0092-259c-4c22-8a56-173428699737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:19.397829 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab0092-259c-4c22-8a56-173428699737 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:19.411487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 361/746] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:39:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:19.411487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab0092-259c-4c22-8a56-173428699737 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:19.457845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b271141-9930-44eb-9821-b9faeaf41e1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:19.459591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b271141-9930-44eb-9821-b9faeaf41e1a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:19.532791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab0092-259c-4c22-8a56-173428699737 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:19.575436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:19.578167 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e3903-8616-4680-bfe6-a5ef8ea8916c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:19.578837 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e3903-8616-4680-bfe6-a5ef8ea8916c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:19.579121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e3903-8616-4680-bfe6-a5ef8ea8916c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:19.579422 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e3903-8616-4680-bfe6-a5ef8ea8916c None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:19.580147 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e3903-8616-4680-bfe6-a5ef8ea8916c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:19.580448 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e3903-8616-4680-bfe6-a5ef8ea8916c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:19.581835 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e3903-8616-4680-bfe6-a5ef8ea8916c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:19.585039 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 386/747] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:39:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:39:19.590266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b271141-9930-44eb-9821-b9faeaf41e1a None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:19.597117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 362/748] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:19.602087 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3476b-aca7-4195-87ec-1b400dfd2b43 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:19.603163 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3476b-aca7-4195-87ec-1b400dfd2b43 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:19.603457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3476b-aca7-4195-87ec-1b400dfd2b43 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:19.609538 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 363/749] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:19.609538 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3476b-aca7-4195-87ec-1b400dfd2b43 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:19.653800 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:19.723794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b271141-9930-44eb-9821-b9faeaf41e1a None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:19.727570 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3476b-aca7-4195-87ec-1b400dfd2b43 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:19.770846 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479eb95b-3561-400c-a5ed-add9a21e8a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:19.771739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eb95b-3561-400c-a5ed-add9a21e8a1e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:19.772013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eb95b-3561-400c-a5ed-add9a21e8a1e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:19.772297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eb95b-3561-400c-a5ed-add9a21e8a1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:19.774553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13139dd8-5855-43c5-b4d0-cca9ec97500a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:19.775761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13139dd8-5855-43c5-b4d0-cca9ec97500a None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:19.776264 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13139dd8-5855-43c5-b4d0-cca9ec97500a None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:19.776771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13139dd8-5855-43c5-b4d0-cca9ec97500a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:19.783564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479eb95b-3561-400c-a5ed-add9a21e8a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:19.785071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479eb95b-3561-400c-a5ed-add9a21e8a1e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:19.789364 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13139dd8-5855-43c5-b4d0-cca9ec97500a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:19.791150 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13139dd8-5855-43c5-b4d0-cca9ec97500a None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:19.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-479eb95b-3561-400c-a5ed-add9a21e8a1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:19.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:19.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:19.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:19.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:19.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:19.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:19.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:19.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:19.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:19.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:19.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:19.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:19.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:19.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:19.806458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:19.807322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:19.807322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:19.807322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:19.807322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:19.807322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 387/750] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:19.807741 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479eb95b-3561-400c-a5ed-add9a21e8a1e None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:19.814546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-13139dd8-5855-43c5-b4d0-cca9ec97500a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:19.814546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:19.814546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:19.814546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:19.814546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:19.814546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:19.814546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:19.814546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:19.814546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:19.814546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:19.814546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:19.814546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:19.814546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:19.814546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:19.814546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:19.815447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:19.816352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:19.816352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:19.816352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:19.816352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:19.822904 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 364/751] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:19.822904 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad11d-ee67-4ecc-8337-dbd68192131a None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:19.822904 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad11d-ee67-4ecc-8337-dbd68192131a None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:19.822904 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad11d-ee67-4ecc-8337-dbd68192131a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:19.839438 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-ed4ad11d-ee67-4ecc-8337-dbd68192131a None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:19.841734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ed4ad11d-ee67-4ecc-8337-dbd68192131a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:19.869284 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:19.929813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 365/752] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:19.930976 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ad11d-ee67-4ecc-8337-dbd68192131a None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:19.947825 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479eb95b-3561-400c-a5ed-add9a21e8a1e None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:19.965723 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:19.998668 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4b0ce-e591-451d-879d-eb2676ea8359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:19.999392 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4b0ce-e591-451d-879d-eb2676ea8359 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:19.999679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4b0ce-e591-451d-879d-eb2676ea8359 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:19.999991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4b0ce-e591-451d-879d-eb2676ea8359 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:20.007604 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4b0ce-e591-451d-879d-eb2676ea8359 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:20.009105 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4b0ce-e591-451d-879d-eb2676ea8359 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:20.023494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-c3d4b0ce-e591-451d-879d-eb2676ea8359 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:20.023494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:20.023494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:20.023494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:20.023494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:20.023494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:20.023494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:20.023494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:20.023494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:20.023494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:20.023494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:20.023494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:20.023494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:20.023494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:20.023494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:20.024613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:20.025575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:20.025575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:20.025575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:20.025575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:20.025575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:20.025575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:20.025575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:20.025575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:20.025575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 388/753] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:20.026086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4b0ce-e591-451d-879d-eb2676ea8359 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:20.068159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ad11d-ee67-4ecc-8337-dbd68192131a None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:20.108084 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e4033-a4a6-4851-85b0-6c2a27acad8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:20.108848 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e4033-a4a6-4851-85b0-6c2a27acad8e None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:20.109308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e4033-a4a6-4851-85b0-6c2a27acad8e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:20.109537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e4033-a4a6-4851-85b0-6c2a27acad8e None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:20.110280 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e4033-a4a6-4851-85b0-6c2a27acad8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:20.110587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e4033-a4a6-4851-85b0-6c2a27acad8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:20.112025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e4033-a4a6-4851-85b0-6c2a27acad8e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:20.126326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 366/754] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:20.127175 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e4033-a4a6-4851-85b0-6c2a27acad8e None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:20.153280 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4b0ce-e591-451d-879d-eb2676ea8359 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:20.161988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:20.194516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0dfb24-8f94-4200-bfef-ce54d920a758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:20.195200 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0dfb24-8f94-4200-bfef-ce54d920a758 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:20.195474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0dfb24-8f94-4200-bfef-ce54d920a758 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:20.195783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0dfb24-8f94-4200-bfef-ce54d920a758 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:20.196517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0dfb24-8f94-4200-bfef-ce54d920a758 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:20.196828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0dfb24-8f94-4200-bfef-ce54d920a758 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:20.198216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0dfb24-8f94-4200-bfef-ce54d920a758 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:20.215944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 389/755] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:20.217908 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0dfb24-8f94-4200-bfef-ce54d920a758 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:20.222684 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:20.246572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e4033-a4a6-4851-85b0-6c2a27acad8e None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:20.288529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f4391-f30a-4b44-a342-cfc5acf75759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:20.289505 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f4391-f30a-4b44-a342-cfc5acf75759 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:20.290163 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f4391-f30a-4b44-a342-cfc5acf75759 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:20.290452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f4391-f30a-4b44-a342-cfc5acf75759 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:20.291388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f4391-f30a-4b44-a342-cfc5acf75759 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:20.293328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f4391-f30a-4b44-a342-cfc5acf75759 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-a56f4391-f30a-4b44-a342-cfc5acf75759 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:20.313149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:20.314235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:20.315113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:20.315113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:20.315113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:20.315113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:20.315113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:20.315113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:20.315113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:20.315113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:20.315113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:20.315113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:20.315113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:20.315113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:20.315113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:20.315113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:39:20.315113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:20.316821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 367/756] 10.222.0.60 () {56 vars in 1129 bytes} [Thu Jul 21 20:39:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:20.316821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f4391-f30a-4b44-a342-cfc5acf75759 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:20.346788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0dfb24-8f94-4200-bfef-ce54d920a758 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:20.399409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b4601-4342-419b-b867-0319874eac2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:20.399409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b4601-4342-419b-b867-0319874eac2a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:20.399409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b4601-4342-419b-b867-0319874eac2a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:20.399409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b4601-4342-419b-b867-0319874eac2a None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:20.399966 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b4601-4342-419b-b867-0319874eac2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:20.399966 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b4601-4342-419b-b867-0319874eac2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:20.402493 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b4601-4342-419b-b867-0319874eac2a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:20.417077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 390/757] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:20.417077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b4601-4342-419b-b867-0319874eac2a None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:20.448150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:20.460034 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f4391-f30a-4b44-a342-cfc5acf75759 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:20.499931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21442b-6512-4322-8928-8ba1167bacd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:20.500806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21442b-6512-4322-8928-8ba1167bacd3 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:20.501090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21442b-6512-4322-8928-8ba1167bacd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:20.501365 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21442b-6512-4322-8928-8ba1167bacd3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:20.509715 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21442b-6512-4322-8928-8ba1167bacd3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:20.511172 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21442b-6512-4322-8928-8ba1167bacd3 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-fa21442b-6512-4322-8928-8ba1167bacd3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:20.527252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:20.528379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:20.529357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:20.529357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:20.529357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:20.529357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:20.529357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:20.529357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:20.529357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:20.529357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:20.529357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:39:20.529357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:20.529357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 368/758] 10.222.0.60 () {56 vars in 1117 bytes} [Thu Jul 21 20:39:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:20.529958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21442b-6512-4322-8928-8ba1167bacd3 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:20.534272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:20.541181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b4601-4342-419b-b867-0319874eac2a None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:20.589164 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f5b82-e4db-4e3c-98b0-d68adf559abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:20.590314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f5b82-e4db-4e3c-98b0-d68adf559abe None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:20.591033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f5b82-e4db-4e3c-98b0-d68adf559abe None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:20.591919 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f5b82-e4db-4e3c-98b0-d68adf559abe None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:20.602177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f5b82-e4db-4e3c-98b0-d68adf559abe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:20.604735 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f5b82-e4db-4e3c-98b0-d68adf559abe None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:20.625503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-5a4f5b82-e4db-4e3c-98b0-d68adf559abe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:20.625503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:20.625503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:20.625503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:20.625503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:20.625503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:20.625503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:20.625503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:20.625503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:20.625503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:20.625503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:20.625503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:20.625503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:20.625503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:20.625503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:20.626457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:20.627325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:20.627325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:20.627325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:20.627325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:20.627325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:20.627325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:20.627325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:20.627325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:20.627325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 391/759] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:20.628738 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f5b82-e4db-4e3c-98b0-d68adf559abe None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:20.665393 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21442b-6512-4322-8928-8ba1167bacd3 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:20.710392 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f7f2c-a76e-43e0-b0f5-39ca8beabcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:20.711078 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f7f2c-a76e-43e0-b0f5-39ca8beabcf1 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:20.711366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f7f2c-a76e-43e0-b0f5-39ca8beabcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:20.711646 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f7f2c-a76e-43e0-b0f5-39ca8beabcf1 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:20.712413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66f7f2c-a76e-43e0-b0f5-39ca8beabcf1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:20.712721 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66f7f2c-a76e-43e0-b0f5-39ca8beabcf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:20.714097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66f7f2c-a76e-43e0-b0f5-39ca8beabcf1 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:20.719467 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 369/760] 10.222.0.60 () {56 vars in 1124 bytes} [Thu Jul 21 20:39:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:20.720146 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f7f2c-a76e-43e0-b0f5-39ca8beabcf1 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:20.749116 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:20.766668 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f5b82-e4db-4e3c-98b0-d68adf559abe None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:20.803394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c01373-3b77-4cb0-a32f-4dcad127c2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:20.804215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c01373-3b77-4cb0-a32f-4dcad127c2d2 None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:20.804502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c01373-3b77-4cb0-a32f-4dcad127c2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:20.804779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c01373-3b77-4cb0-a32f-4dcad127c2d2 None admin] PATH_INFO: `/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/77812b356c974d2b80102217956169db/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:20.807238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c01373-3b77-4cb0-a32f-4dcad127c2d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc61fe15894891a9054911321f4ac3, user_id=77812b356c974d2b80102217956169db, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:20.808731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c01373-3b77-4cb0-a32f-4dcad127c2d2 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:20.810586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:20.831749 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 392/761] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:39:20 2022] PUT /identity/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/77812b356c974d2b80102217956169db/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:20.831749 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c01373-3b77-4cb0-a32f-4dcad127c2d2 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:20.870133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f7f2c-a76e-43e0-b0f5-39ca8beabcf1 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:20.923765 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7d721-c4b6-4613-ae30-2d1698b67312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:20.924741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7d721-c4b6-4613-ae30-2d1698b67312 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:20.925030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7d721-c4b6-4613-ae30-2d1698b67312 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:20.925310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7d721-c4b6-4613-ae30-2d1698b67312 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:20.926354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7d721-c4b6-4613-ae30-2d1698b67312 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:20.927049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7d721-c4b6-4613-ae30-2d1698b67312 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:20.928739 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7d721-c4b6-4613-ae30-2d1698b67312 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:20.944106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c01373-3b77-4cb0-a32f-4dcad127c2d2 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:20.955921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 370/762] 10.222.0.60 () {56 vars in 1149 bytes} [Thu Jul 21 20:39:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:20.956833 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7d721-c4b6-4613-ae30-2d1698b67312 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:20.996486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06fa81-d816-494d-926e-2bb260f1bc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:20.997320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06fa81-d816-494d-926e-2bb260f1bc77 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:20.997614 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06fa81-d816-494d-926e-2bb260f1bc77 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:20.997937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06fa81-d816-494d-926e-2bb260f1bc77 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:20.998690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06fa81-d816-494d-926e-2bb260f1bc77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:20.999006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06fa81-d816-494d-926e-2bb260f1bc77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:21.000496 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06fa81-d816-494d-926e-2bb260f1bc77 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:21.019255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 393/763] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:21.021878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06fa81-d816-494d-926e-2bb260f1bc77 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:21.033611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:21.105143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7d721-c4b6-4613-ae30-2d1698b67312 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:21.141444 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:21.149260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06fa81-d816-494d-926e-2bb260f1bc77 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:21.154504 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb263ea-68a2-46d1-a46f-013886067df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:21.155191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb263ea-68a2-46d1-a46f-013886067df5 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:21.155474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb263ea-68a2-46d1-a46f-013886067df5 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:21.155772 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb263ea-68a2-46d1-a46f-013886067df5 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:21.156507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb263ea-68a2-46d1-a46f-013886067df5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:21.156811 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb263ea-68a2-46d1-a46f-013886067df5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:21.158780 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb263ea-68a2-46d1-a46f-013886067df5 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:21.171291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 371/764] 10.222.0.60 () {56 vars in 1124 bytes} [Thu Jul 21 20:39:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:21.172200 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb263ea-68a2-46d1-a46f-013886067df5 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:21.199754 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c383a-a98e-452d-9990-daa7a0990c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:21.200492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c383a-a98e-452d-9990-daa7a0990c4f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:21.200790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c383a-a98e-452d-9990-daa7a0990c4f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:21.201072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c383a-a98e-452d-9990-daa7a0990c4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:21.289107 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c383a-a98e-452d-9990-daa7a0990c4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:21.291776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c383a-a98e-452d-9990-daa7a0990c4f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:21.303334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:21.316060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb263ea-68a2-46d1-a46f-013886067df5 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:21.363014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd1bd9-3122-4de6-b7cc-12c91458eba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:21.363601 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd1bd9-3122-4de6-b7cc-12c91458eba3 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:21.363978 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd1bd9-3122-4de6-b7cc-12c91458eba3 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:21.366180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd1bd9-3122-4de6-b7cc-12c91458eba3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:21.366180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd1bd9-3122-4de6-b7cc-12c91458eba3 None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=833a474f828f4349bea0d1649461c77e) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:21.366412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd1bd9-3122-4de6-b7cc-12c91458eba3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:21.367830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd1bd9-3122-4de6-b7cc-12c91458eba3 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:21.380154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 372/765] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:39:21 2022] GET /identity/v3/role_assignments?user.id=833a474f828f4349bea0d1649461c77e&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:21.380154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd1bd9-3122-4de6-b7cc-12c91458eba3 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:21.449611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 394/766] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:39:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:39:21.451943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019b910-f378-4067-a505-883a7a04c749 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:21.452228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019b910-f378-4067-a505-883a7a04c749 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:21.452503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019b910-f378-4067-a505-883a7a04c749 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:21.455919 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 395/767] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:21.457667 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019b910-f378-4067-a505-883a7a04c749 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:21.462507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:21.522968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd1bd9-3122-4de6-b7cc-12c91458eba3 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:21.572895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2e732-e5ca-450d-85a1-925cd41be721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:21.573999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2e732-e5ca-450d-85a1-925cd41be721 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:21.574465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2e732-e5ca-450d-85a1-925cd41be721 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:21.574788 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2e732-e5ca-450d-85a1-925cd41be721 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:21.576456 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2e732-e5ca-450d-85a1-925cd41be721 None admin] RBAC: Adding query filter params (interface=public, service_id=ea0f8bfa4f92463f8a0e461f94f30b20, region_id=RegionOne) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:21.576765 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2e732-e5ca-450d-85a1-925cd41be721 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:21.578264 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2e732-e5ca-450d-85a1-925cd41be721 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:21.589528 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 373/768] 10.222.0.60 () {56 vars in 1251 bytes} [Thu Jul 21 20:39:21 2022] GET /identity/v3/endpoints?service_id=ea0f8bfa4f92463f8a0e461f94f30b20&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:21.590975 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9c4ec-5aca-44fe-966b-9d71d770e185 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:21.591261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9c4ec-5aca-44fe-966b-9d71d770e185 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:21.592407 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9c4ec-5aca-44fe-966b-9d71d770e185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:21.611645 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-3db9c4ec-5aca-44fe-966b-9d71d770e185 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:21.612004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019b910-f378-4067-a505-883a7a04c749 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:21.612992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-3db9c4ec-5aca-44fe-966b-9d71d770e185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:21.620820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:21.655834 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084a398-7e49-4e20-8691-5931780fba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:21.656743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084a398-7e49-4e20-8691-5931780fba67 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:21.657131 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084a398-7e49-4e20-8691-5931780fba67 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:21.657416 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084a398-7e49-4e20-8691-5931780fba67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:21.672776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084a398-7e49-4e20-8691-5931780fba67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:21.675476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084a398-7e49-4e20-8691-5931780fba67 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:21.697422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-6084a398-7e49-4e20-8691-5931780fba67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:21.697422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:21.697422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:21.697422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:21.697422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:21.697422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:21.697422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:21.697422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:21.697422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:21.697422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:21.697422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:21.697422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:21.697422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:21.697422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:21.697422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:21.698382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:21.699247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:21.699247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:21.699247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:21.699247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:21.699247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 396/769] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:21.700422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084a398-7e49-4e20-8691-5931780fba67 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:21.706586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 374/770] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:21.707573 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9c4ec-5aca-44fe-966b-9d71d770e185 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:21.772655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:21.841741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9c4ec-5aca-44fe-966b-9d71d770e185 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:21.846682 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084a398-7e49-4e20-8691-5931780fba67 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:21.888938 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a194f12-d6c8-4909-a6ff-ee1837749e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:21.889952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a194f12-d6c8-4909-a6ff-ee1837749e23 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:21.890632 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a194f12-d6c8-4909-a6ff-ee1837749e23 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:21.891146 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a194f12-d6c8-4909-a6ff-ee1837749e23 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:21.891957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a042c59-ee1f-4189-bf8a-fb92c1e011f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:21.892092 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a194f12-d6c8-4909-a6ff-ee1837749e23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:21.892543 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a194f12-d6c8-4909-a6ff-ee1837749e23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:21.893182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a042c59-ee1f-4189-bf8a-fb92c1e011f2 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:21.893638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a042c59-ee1f-4189-bf8a-fb92c1e011f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:21.894066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a042c59-ee1f-4189-bf8a-fb92c1e011f2 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:21.894335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a194f12-d6c8-4909-a6ff-ee1837749e23 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:21.895054 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a042c59-ee1f-4189-bf8a-fb92c1e011f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:21.896946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a042c59-ee1f-4189-bf8a-fb92c1e011f2 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:21.906800 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 397/771] 10.222.0.60 () {56 vars in 1087 bytes} [Thu Jul 21 20:39:21 2022] GET /identity/v3/services => generated 772 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:21.908045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a042c59-ee1f-4189-bf8a-fb92c1e011f2 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:21.908704 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 375/772] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:21.921416 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a194f12-d6c8-4909-a6ff-ee1837749e23 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:21.945252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:22.043965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a042c59-ee1f-4189-bf8a-fb92c1e011f2 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:22.085828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66b308-ab85-4b47-9b78-1a8f098ec9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:22.087038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66b308-ab85-4b47-9b78-1a8f098ec9db None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:22.087579 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66b308-ab85-4b47-9b78-1a8f098ec9db None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:22.087984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66b308-ab85-4b47-9b78-1a8f098ec9db None admin] PATH_INFO: `/v3/services/network` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:22.088761 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66b308-ab85-4b47-9b78-1a8f098ec9db None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:22.090247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66b308-ab85-4b47-9b78-1a8f098ec9db None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-4e66b308-ab85-4b47-9b78-1a8f098ec9db None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:22.103939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:22.105471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:22.106294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:22.106294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:22.106294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:22.106294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:22.106294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:22.106294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:22.106294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:22.106294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:22.106294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:22.106294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:22.106294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:22.106294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:22.106294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:39:22.106294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:22.111482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 398/773] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:21 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:22.112912 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66b308-ab85-4b47-9b78-1a8f098ec9db None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:22.151972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a194f12-d6c8-4909-a6ff-ee1837749e23 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:22.165008 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:22.207669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1692a-64e0-4be6-a7cb-ca1c4c09d510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:22.209019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1692a-64e0-4be6-a7cb-ca1c4c09d510 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:22.209549 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1692a-64e0-4be6-a7cb-ca1c4c09d510 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:22.210034 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1692a-64e0-4be6-a7cb-ca1c4c09d510 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:22.219835 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1692a-64e0-4be6-a7cb-ca1c4c09d510 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:22.221933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1692a-64e0-4be6-a7cb-ca1c4c09d510 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:22.227228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:22.245233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66b308-ab85-4b47-9b78-1a8f098ec9db None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:22.248355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-24e1692a-64e0-4be6-a7cb-ca1c4c09d510 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:22.248355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:22.248355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:22.248355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:22.248355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:22.248355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:22.248355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:22.248355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:22.248355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:22.248355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:22.248355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:22.248355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:22.248355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:22.248355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:22.248355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:22.252843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:22.254054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:22.254054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:22.254054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:22.254054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:22.254054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:22.254054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:22.254054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:22.254054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:22.254054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 376/774] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:22.264315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1692a-64e0-4be6-a7cb-ca1c4c09d510 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:22.341341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a1277-c429-4ad4-b56c-2d2e26dc3801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:22.342456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a1277-c429-4ad4-b56c-2d2e26dc3801 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:22.342949 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a1277-c429-4ad4-b56c-2d2e26dc3801 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:22.343420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a1277-c429-4ad4-b56c-2d2e26dc3801 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:22.344344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a1277-c429-4ad4-b56c-2d2e26dc3801 None admin] RBAC: Adding query filter params (name=network) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:22.344836 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a1277-c429-4ad4-b56c-2d2e26dc3801 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:22.346415 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a1277-c429-4ad4-b56c-2d2e26dc3801 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:22.358395 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 399/775] 10.222.0.60 () {56 vars in 1112 bytes} [Thu Jul 21 20:39:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:22.364676 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a1277-c429-4ad4-b56c-2d2e26dc3801 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:22.487113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1692a-64e0-4be6-a7cb-ca1c4c09d510 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:22.529201 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2e43f-d8b0-47e7-8115-7da01a62ce4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:22.529884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2e43f-d8b0-47e7-8115-7da01a62ce4a None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:22.530170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2e43f-d8b0-47e7-8115-7da01a62ce4a None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:22.530446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2e43f-d8b0-47e7-8115-7da01a62ce4a None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:22.531257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2e43f-d8b0-47e7-8115-7da01a62ce4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:22.531567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2e43f-d8b0-47e7-8115-7da01a62ce4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:22.533073 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2e43f-d8b0-47e7-8115-7da01a62ce4a None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:22.568817 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 377/776] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:22.575404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2e43f-d8b0-47e7-8115-7da01a62ce4a None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:22.615671 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a1277-c429-4ad4-b56c-2d2e26dc3801 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:22.639256 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:22.662511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ce6fb-08fa-4db1-b711-a6abc90e010e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:22.663120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ce6fb-08fa-4db1-b711-a6abc90e010e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:22.663412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ce6fb-08fa-4db1-b711-a6abc90e010e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:22.663691 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ce6fb-08fa-4db1-b711-a6abc90e010e None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:22.664452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ce6fb-08fa-4db1-b711-a6abc90e010e None admin] RBAC: Adding query filter params (type=network) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:22.664762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ce6fb-08fa-4db1-b711-a6abc90e010e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:22.666332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ce6fb-08fa-4db1-b711-a6abc90e010e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:22.679973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 400/777] 10.222.0.60 () {56 vars in 1112 bytes} [Thu Jul 21 20:39:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:22.681461 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ce6fb-08fa-4db1-b711-a6abc90e010e None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:22.709887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:22.817405 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2e43f-d8b0-47e7-8115-7da01a62ce4a None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:22.889018 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45dfbc-a577-46a9-8c09-c7abc876689f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:22.890195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45dfbc-a577-46a9-8c09-c7abc876689f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:22.891987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45dfbc-a577-46a9-8c09-c7abc876689f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:22.894103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ce6fb-08fa-4db1-b711-a6abc90e010e None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:22.895009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45dfbc-a577-46a9-8c09-c7abc876689f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:22.896761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d45dfbc-a577-46a9-8c09-c7abc876689f None admin] RBAC: Adding query filter params (role.id=05f3681d0a644607934e281f75ecd675, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=f0beb65e5dc7469eb17a98dae1f09a0e) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:22.899002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d45dfbc-a577-46a9-8c09-c7abc876689f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:22.900726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d45dfbc-a577-46a9-8c09-c7abc876689f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:22.920933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 378/778] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:39:22 2022] GET /identity/v3/role_assignments?user.id=f0beb65e5dc7469eb17a98dae1f09a0e&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=05f3681d0a644607934e281f75ecd675 => generated 259 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:22.956610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b5e0d-ff3c-49fb-b333-ed6121d16862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:22.957747 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b5e0d-ff3c-49fb-b333-ed6121d16862 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:22.958655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b5e0d-ff3c-49fb-b333-ed6121d16862 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:22.959164 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b5e0d-ff3c-49fb-b333-ed6121d16862 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:22.960216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b5e0d-ff3c-49fb-b333-ed6121d16862 None admin] RBAC: Adding query filter params (interface=public, service_id=c90a6e06073e41e29074e47ecba0b731, region_id=RegionOne) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:22.960758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b5e0d-ff3c-49fb-b333-ed6121d16862 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:22.962502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b5e0d-ff3c-49fb-b333-ed6121d16862 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:22.971940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 401/779] 10.222.0.60 () {56 vars in 1251 bytes} [Thu Jul 21 20:39:22 2022] GET /identity/v3/endpoints?service_id=c90a6e06073e41e29074e47ecba0b731&interface=public®ion_id=RegionOne => generated 194 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:22.981033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45dfbc-a577-46a9-8c09-c7abc876689f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:23.124106 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:23.128592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45dfbc-a577-46a9-8c09-c7abc876689f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:23.153381 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db4c90-21a5-48f9-9a32-fe09b19f6be0 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:23.153722 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db4c90-21a5-48f9-9a32-fe09b19f6be0 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:23.153956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db4c90-21a5-48f9-9a32-fe09b19f6be0 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:23.155657 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 402/780] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:23.161753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfb447-9b74-4c56-a1c3-b106579df1b5 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:23.162213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfb447-9b74-4c56-a1c3-b106579df1b5 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:23.162724 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfb447-9b74-4c56-a1c3-b106579df1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:23.183042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-b5cfb447-9b74-4c56-a1c3-b106579df1b5 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:23.184279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b5cfb447-9b74-4c56-a1c3-b106579df1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:23.187310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ec47a-e209-43f9-8b4a-ceede24cd3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:23.191694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ec47a-e209-43f9-8b4a-ceede24cd3b6 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:23.191694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ec47a-e209-43f9-8b4a-ceede24cd3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:23.191694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ec47a-e209-43f9-8b4a-ceede24cd3b6 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:23.191694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ec47a-e209-43f9-8b4a-ceede24cd3b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:23.191694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ec47a-e209-43f9-8b4a-ceede24cd3b6 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:23.203271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 379/781] 10.222.0.60 () {56 vars in 1087 bytes} [Thu Jul 21 20:39:22 2022] GET /identity/v3/services => generated 772 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:23.303607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 403/782] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:23.341224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ec47a-e209-43f9-8b4a-ceede24cd3b6 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:23.480406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ec47a-e209-43f9-8b4a-ceede24cd3b6 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:23.518768 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937b8c6-fdb3-4230-89fa-cd87010f97e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:23.521454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937b8c6-fdb3-4230-89fa-cd87010f97e4 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:23.522190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937b8c6-fdb3-4230-89fa-cd87010f97e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:23.522684 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937b8c6-fdb3-4230-89fa-cd87010f97e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:23.539049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937b8c6-fdb3-4230-89fa-cd87010f97e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-0937b8c6-fdb3-4230-89fa-cd87010f97e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:23.549695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:23.552348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:23.553390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:23.553390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:23.553390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:23.553390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:23.553390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:23.553390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:23.553390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:23.553390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:23.553390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:23.553390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:23.553390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:39:23.553390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:23.555780 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 380/783] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:39:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:23.567780 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfb447-9b74-4c56-a1c3-b106579df1b5 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:23.578180 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:23.734956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfb447-9b74-4c56-a1c3-b106579df1b5 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:23.783921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee158552-b083-43b7-b2ae-0bbe5d414cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:23.784619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee158552-b083-43b7-b2ae-0bbe5d414cbd None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:23.784905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee158552-b083-43b7-b2ae-0bbe5d414cbd None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:23.785185 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee158552-b083-43b7-b2ae-0bbe5d414cbd None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:23.786658 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee158552-b083-43b7-b2ae-0bbe5d414cbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:23.786905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee158552-b083-43b7-b2ae-0bbe5d414cbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:23.788306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee158552-b083-43b7-b2ae-0bbe5d414cbd None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:23.802871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 404/784] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:39:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:23.808139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937b8c6-fdb3-4230-89fa-cd87010f97e4 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:23.850521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:23.897864 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc079a1-448d-4589-add5-dbbfb4498d41 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:23.898437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc079a1-448d-4589-add5-dbbfb4498d41 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:23.898437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc079a1-448d-4589-add5-dbbfb4498d41 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:23.900136 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 405/785] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:39:23.911956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb278dcd-5df5-43a6-b531-0e14abffbbc7 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:23.912333 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb278dcd-5df5-43a6-b531-0e14abffbbc7 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:23.912626 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb278dcd-5df5-43a6-b531-0e14abffbbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:23.944391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-bb278dcd-5df5-43a6-b531-0e14abffbbc7 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:23.945621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-bb278dcd-5df5-43a6-b531-0e14abffbbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:24.016251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937b8c6-fdb3-4230-89fa-cd87010f97e4 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:24.016939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:24.022907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 406/786] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:24.046660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb278dcd-5df5-43a6-b531-0e14abffbbc7 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:24.082067 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f36b5c-2ef9-4a70-ba84-28397f5d492c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:24.083532 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f36b5c-2ef9-4a70-ba84-28397f5d492c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:24.084080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f36b5c-2ef9-4a70-ba84-28397f5d492c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:24.084597 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f36b5c-2ef9-4a70-ba84-28397f5d492c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:24.105833 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f36b5c-2ef9-4a70-ba84-28397f5d492c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:24.107749 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f36b5c-2ef9-4a70-ba84-28397f5d492c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:24.139838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-47f36b5c-2ef9-4a70-ba84-28397f5d492c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:24.139838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:24.139838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:24.139838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:24.139838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:24.139838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:24.139838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:24.139838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:24.139838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:24.139838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:24.139838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:24.139838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:24.139838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:24.139838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:24.139838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:24.140827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:24.141693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:24.141693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:24.141693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:24.141693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:24.146987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 381/787] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:24.152328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f36b5c-2ef9-4a70-ba84-28397f5d492c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:24.170350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb278dcd-5df5-43a6-b531-0e14abffbbc7 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:24.210324 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1d3a8-3a27-410c-bdcf-4725ddfbb052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:24.211534 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1d3a8-3a27-410c-bdcf-4725ddfbb052 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:24.212048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1d3a8-3a27-410c-bdcf-4725ddfbb052 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:24.212533 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1d3a8-3a27-410c-bdcf-4725ddfbb052 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:24.225036 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1d3a8-3a27-410c-bdcf-4725ddfbb052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-16d1d3a8-3a27-410c-bdcf-4725ddfbb052 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:24.233435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:24.234402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:24.235219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:24.235219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:24.235219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:24.235219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:24.235219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:24.235219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:24.235219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:24.235219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:24.235219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:24.235219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:24.235219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:39:24.235219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:24.238022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 407/788] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:39:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:24.245008 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1d3a8-3a27-410c-bdcf-4725ddfbb052 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:24.332133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:24.344125 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:24.360077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f36b5c-2ef9-4a70-ba84-28397f5d492c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:24.363099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1d3a8-3a27-410c-bdcf-4725ddfbb052 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:24.419461 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fe78a-5062-4bb1-a1ca-2f37007117d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:24.420438 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fe78a-5062-4bb1-a1ca-2f37007117d0 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:24.420800 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fe78a-5062-4bb1-a1ca-2f37007117d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:24.426753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fe78a-5062-4bb1-a1ca-2f37007117d0 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:24.426753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36fe78a-5062-4bb1-a1ca-2f37007117d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:24.426753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36fe78a-5062-4bb1-a1ca-2f37007117d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:24.426753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36fe78a-5062-4bb1-a1ca-2f37007117d0 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:24.436273 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa63a7-f990-4ef7-8b61-7ecd63097eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:24.437013 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa63a7-f990-4ef7-8b61-7ecd63097eef None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:24.437307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa63a7-f990-4ef7-8b61-7ecd63097eef None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:24.437587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa63a7-f990-4ef7-8b61-7ecd63097eef None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:24.438328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa63a7-f990-4ef7-8b61-7ecd63097eef None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:24.438634 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa63a7-f990-4ef7-8b61-7ecd63097eef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:24.440054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa63a7-f990-4ef7-8b61-7ecd63097eef None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:24.446153 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 408/789] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:39:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:24.457048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fe78a-5062-4bb1-a1ca-2f37007117d0 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:24.462276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 382/790] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:24.467497 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa63a7-f990-4ef7-8b61-7ecd63097eef None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:24.612448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fe78a-5062-4bb1-a1ca-2f37007117d0 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:24.641040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:24.657861 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410d39c-dbcd-4974-a1dd-7c172a35644a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:24.658909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410d39c-dbcd-4974-a1dd-7c172a35644a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:24.660164 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410d39c-dbcd-4974-a1dd-7c172a35644a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:24.660721 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410d39c-dbcd-4974-a1dd-7c172a35644a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:24.690102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410d39c-dbcd-4974-a1dd-7c172a35644a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:24.691637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410d39c-dbcd-4974-a1dd-7c172a35644a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:24.699258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa63a7-f990-4ef7-8b61-7ecd63097eef None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:24.721276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-a410d39c-dbcd-4974-a1dd-7c172a35644a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:24.721276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:24.721276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:24.721276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:24.721276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:24.721276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:24.721276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:24.721276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:24.721276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:24.721276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:24.721276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:24.721276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:24.721276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:24.721276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:24.721276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:24.722259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:24.723112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:24.723112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:24.723112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:24.723112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:24.732187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 409/791] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:24.734088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:24.738153 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410d39c-dbcd-4974-a1dd-7c172a35644a None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:24.748049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680ec2e-18d5-473b-b5ad-b05dbe32c45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:24.749419 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680ec2e-18d5-473b-b5ad-b05dbe32c45b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:24.749702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680ec2e-18d5-473b-b5ad-b05dbe32c45b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:24.749975 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680ec2e-18d5-473b-b5ad-b05dbe32c45b None admin] PATH_INFO: `/v3/users/nova` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:24.759246 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c680ec2e-18d5-473b-b5ad-b05dbe32c45b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:24.760983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c680ec2e-18d5-473b-b5ad-b05dbe32c45b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-c680ec2e-18d5-473b-b5ad-b05dbe32c45b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:24.779139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:24.780372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:24.781295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:24.781295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:24.781295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:24.781295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:24.781295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:24.781295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:24.781295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:24.781295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:24.781295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:39:24.781295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:24.781295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 383/792] 10.222.0.60 () {56 vars in 1093 bytes} [Thu Jul 21 20:39:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:24.791511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b861f-a5e0-410d-aca2-de0fb6f44254 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:24.791511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b861f-a5e0-410d-aca2-de0fb6f44254 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:24.791511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b861f-a5e0-410d-aca2-de0fb6f44254 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:24.791511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 384/793] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:24.791511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b861f-a5e0-410d-aca2-de0fb6f44254 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:24.883426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410d39c-dbcd-4974-a1dd-7c172a35644a None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:24.932192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6acd3-d5d0-4f64-865a-8f79193d766e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:24.933806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6acd3-d5d0-4f64-865a-8f79193d766e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:24.934306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6acd3-d5d0-4f64-865a-8f79193d766e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:24.934417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b861f-a5e0-410d-aca2-de0fb6f44254 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:24.935274 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6acd3-d5d0-4f64-865a-8f79193d766e None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:24.936323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6acd3-d5d0-4f64-865a-8f79193d766e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:24.937225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6acd3-d5d0-4f64-865a-8f79193d766e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:24.940126 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6acd3-d5d0-4f64-865a-8f79193d766e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:24.958549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 410/794] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:24.961008 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a66b6-d5ba-4de7-a240-58a349dc8532 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:24.961527 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a66b6-d5ba-4de7-a240-58a349dc8532 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:24.962044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a66b6-d5ba-4de7-a240-58a349dc8532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:24.982890 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35790d36-c90c-4033-9b68-c70350dd3759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:24.983589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35790d36-c90c-4033-9b68-c70350dd3759 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:24.983933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35790d36-c90c-4033-9b68-c70350dd3759 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:24.984216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35790d36-c90c-4033-9b68-c70350dd3759 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:24.985133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35790d36-c90c-4033-9b68-c70350dd3759 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:24.985448 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35790d36-c90c-4033-9b68-c70350dd3759 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:24.986291 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-c75a66b6-d5ba-4de7-a240-58a349dc8532 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:24.986956 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35790d36-c90c-4033-9b68-c70350dd3759 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:24.987971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c75a66b6-d5ba-4de7-a240-58a349dc8532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:25.011791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 385/795] 10.222.0.60 () {56 vars in 1133 bytes} [Thu Jul 21 20:39:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:25.011791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35790d36-c90c-4033-9b68-c70350dd3759 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:25.032492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:25.074951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:25.092026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 411/796] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:25.093351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a66b6-d5ba-4de7-a240-58a349dc8532 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:25.167331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35790d36-c90c-4033-9b68-c70350dd3759 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:25.215466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d2fca-5d25-408b-a03b-dc76535bda87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:25.216685 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d2fca-5d25-408b-a03b-dc76535bda87 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:25.217430 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d2fca-5d25-408b-a03b-dc76535bda87 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:25.217957 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d2fca-5d25-408b-a03b-dc76535bda87 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:25.221757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a66b6-d5ba-4de7-a240-58a349dc8532 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:25.228990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d2fca-5d25-408b-a03b-dc76535bda87 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:25.231625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d2fca-5d25-408b-a03b-dc76535bda87 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-299d2fca-5d25-408b-a03b-dc76535bda87 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:25.252319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:25.253331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:25.254299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:25.254299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:25.254299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:25.254299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:25.254299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:25.254299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:25.254299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:25.254299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:25.254299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:39:25.254299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:25.254299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 386/797] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:39:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:25.255798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d2fca-5d25-408b-a03b-dc76535bda87 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:25.276304 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeda228-59ec-4ee0-82d2-6476b41690b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:25.277623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeda228-59ec-4ee0-82d2-6476b41690b3 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:25.278176 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeda228-59ec-4ee0-82d2-6476b41690b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:25.278928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeda228-59ec-4ee0-82d2-6476b41690b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:25.327638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:25.375325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeda228-59ec-4ee0-82d2-6476b41690b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:25.376526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d2fca-5d25-408b-a03b-dc76535bda87 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:25.376819 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeda228-59ec-4ee0-82d2-6476b41690b3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:25.387123 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:25.418710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c938f6-dd1d-4bef-830b-551bf8e2ec3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:25.419453 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c938f6-dd1d-4bef-830b-551bf8e2ec3c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:25.419858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c938f6-dd1d-4bef-830b-551bf8e2ec3c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:25.420033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c938f6-dd1d-4bef-830b-551bf8e2ec3c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:25.420688 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c938f6-dd1d-4bef-830b-551bf8e2ec3c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:25.422075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c938f6-dd1d-4bef-830b-551bf8e2ec3c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-a2c938f6-dd1d-4bef-830b-551bf8e2ec3c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:25.434830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:25.435960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:25.444396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 387/798] 10.222.0.60 () {56 vars in 1129 bytes} [Thu Jul 21 20:39:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:25.445250 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c938f6-dd1d-4bef-830b-551bf8e2ec3c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:25.523139 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 412/799] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:39:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:39:25.525041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fec26d-037e-4115-b4d7-b2acc9d5209a None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:25.525710 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fec26d-037e-4115-b4d7-b2acc9d5209a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:25.526211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fec26d-037e-4115-b4d7-b2acc9d5209a None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:25.527940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 413/800] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:25.528930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fec26d-037e-4115-b4d7-b2acc9d5209a None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:25.579140 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c938f6-dd1d-4bef-830b-551bf8e2ec3c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:25.611062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:25.618295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf1c1f-283e-4907-a757-1c7fca774934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:25.618984 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf1c1f-283e-4907-a757-1c7fca774934 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:25.619270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf1c1f-283e-4907-a757-1c7fca774934 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:25.619550 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf1c1f-283e-4907-a757-1c7fca774934 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:25.620439 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf1c1f-283e-4907-a757-1c7fca774934 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:25.623069 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf1c1f-283e-4907-a757-1c7fca774934 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:25.623069 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf1c1f-283e-4907-a757-1c7fca774934 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:25.651922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 388/801] 10.222.0.60 () {56 vars in 1135 bytes} [Thu Jul 21 20:39:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:25.652353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf1c1f-283e-4907-a757-1c7fca774934 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:25.670003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fec26d-037e-4115-b4d7-b2acc9d5209a None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:25.683735 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:25.712323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b3daf-cff7-4ab3-a545-0bae0d09f642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:25.712995 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b3daf-cff7-4ab3-a545-0bae0d09f642 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:25.713279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b3daf-cff7-4ab3-a545-0bae0d09f642 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:25.713559 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b3daf-cff7-4ab3-a545-0bae0d09f642 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:25.714276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b3daf-cff7-4ab3-a545-0bae0d09f642 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:25.714597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b3daf-cff7-4ab3-a545-0bae0d09f642 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:25.716012 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b3daf-cff7-4ab3-a545-0bae0d09f642 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:25.720853 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 414/802] 10.222.0.60 () {56 vars in 1124 bytes} [Thu Jul 21 20:39:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:25.722085 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a3f47-3a8a-47e8-95df-885aeb75c994 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:25.727937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a3f47-3a8a-47e8-95df-885aeb75c994 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:25.728245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a3f47-3a8a-47e8-95df-885aeb75c994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:25.747007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-c03a3f47-3a8a-47e8-95df-885aeb75c994 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:25.748268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c03a3f47-3a8a-47e8-95df-885aeb75c994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:25.778183 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf1c1f-283e-4907-a757-1c7fca774934 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:25.819854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ea9b1-6df4-4b70-aeac-3a99baf406fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:25.821005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ea9b1-6df4-4b70-aeac-3a99baf406fb None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:25.821480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ea9b1-6df4-4b70-aeac-3a99baf406fb None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:25.821830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ea9b1-6df4-4b70-aeac-3a99baf406fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:25.835911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 415/803] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:25.835911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a3f47-3a8a-47e8-95df-885aeb75c994 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:25.843047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ea9b1-6df4-4b70-aeac-3a99baf406fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:25.845367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ea9b1-6df4-4b70-aeac-3a99baf406fb None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:25.869521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-5f4ea9b1-6df4-4b70-aeac-3a99baf406fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:25.869521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:25.869521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:25.869521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:25.869521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:25.869521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:25.869521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:25.869521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:25.869521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:25.869521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:25.869521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:25.869521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:25.869521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:25.869521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:25.869521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:25.870488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:25.871379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:25.871379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:25.871379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:25.871379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:25.871379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 389/804] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:25.871669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ea9b1-6df4-4b70-aeac-3a99baf406fb None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:25.923453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:25.962565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:25.977665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a3f47-3a8a-47e8-95df-885aeb75c994 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:25.985710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ea9b1-6df4-4b70-aeac-3a99baf406fb None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:26.015924 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f97b4-cc07-414a-8f1c-0c7b8d7645ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:26.016728 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f97b4-cc07-414a-8f1c-0c7b8d7645ea None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.017069 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f97b4-cc07-414a-8f1c-0c7b8d7645ea None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.017287 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f97b4-cc07-414a-8f1c-0c7b8d7645ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.021325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983e58d-e431-4d12-acde-e74ce0f9397f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:26.022006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983e58d-e431-4d12-acde-e74ce0f9397f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.022300 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983e58d-e431-4d12-acde-e74ce0f9397f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.022569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983e58d-e431-4d12-acde-e74ce0f9397f None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.023290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983e58d-e431-4d12-acde-e74ce0f9397f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:26.023599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983e58d-e431-4d12-acde-e74ce0f9397f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:26.025041 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983e58d-e431-4d12-acde-e74ce0f9397f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:26.028753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f97b4-cc07-414a-8f1c-0c7b8d7645ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:26.030207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 390/805] 10.222.0.60 () {56 vars in 1124 bytes} [Thu Jul 21 20:39:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:26.030345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f97b4-cc07-414a-8f1c-0c7b8d7645ea None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:26.031015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983e58d-e431-4d12-acde-e74ce0f9397f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:26.048889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-da2f97b4-cc07-414a-8f1c-0c7b8d7645ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:26.048889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:26.048889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:26.048889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:26.048889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:26.048889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:26.048889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:26.048889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:26.048889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:26.048889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:26.048889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:26.048889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:26.048889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:26.048889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:26.048889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:26.049900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:26.050781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:26.050781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:26.050781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:26.050781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:26.050781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 416/806] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:26.051060 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f97b4-cc07-414a-8f1c-0c7b8d7645ea None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:26.154092 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983e58d-e431-4d12-acde-e74ce0f9397f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:26.168222 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f97b4-cc07-414a-8f1c-0c7b8d7645ea None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:26.180173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:26.192415 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967eb07-ee6e-40bd-a2c6-968f596e8c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:26.193226 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967eb07-ee6e-40bd-a2c6-968f596e8c7b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.193511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967eb07-ee6e-40bd-a2c6-968f596e8c7b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.193793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967eb07-ee6e-40bd-a2c6-968f596e8c7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.207602 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967eb07-ee6e-40bd-a2c6-968f596e8c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:26.209265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e5bf0-3ccf-4d73-91b7-e0dadd9b6b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:26.209943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e5bf0-3ccf-4d73-91b7-e0dadd9b6b35 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.210221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e5bf0-3ccf-4d73-91b7-e0dadd9b6b35 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.210495 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e5bf0-3ccf-4d73-91b7-e0dadd9b6b35 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.212934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e5bf0-3ccf-4d73-91b7-e0dadd9b6b35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:26.212934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e5bf0-3ccf-4d73-91b7-e0dadd9b6b35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:26.212934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e5bf0-3ccf-4d73-91b7-e0dadd9b6b35 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-3967eb07-ee6e-40bd-a2c6-968f596e8c7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:26.215840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:26.216924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:26.217839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:26.217839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:26.217839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:26.217839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:26.217839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:26.217839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:26.217839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:26.217839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:26.217839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:26.217839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:26.217839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:26.217839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:26.217839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 391/807] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:39:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:26.218524 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264af19-bac2-4995-ac3e-db224e2d6ca0 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.219074 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264af19-bac2-4995-ac3e-db224e2d6ca0 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.219074 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264af19-bac2-4995-ac3e-db224e2d6ca0 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.222651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 392/808] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:26.222651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264af19-bac2-4995-ac3e-db224e2d6ca0 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:26.225965 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:26.232489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 417/809] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:26.232489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e5bf0-3ccf-4d73-91b7-e0dadd9b6b35 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:26.341914 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e5bf0-3ccf-4d73-91b7-e0dadd9b6b35 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:26.346383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264af19-bac2-4995-ac3e-db224e2d6ca0 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:26.382430 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4881f25-d0b3-477c-a63c-978d7d5713f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:26.383500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4881f25-d0b3-477c-a63c-978d7d5713f2 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.384021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4881f25-d0b3-477c-a63c-978d7d5713f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.384466 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4881f25-d0b3-477c-a63c-978d7d5713f2 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.384709 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad7b6e-1ed1-439b-8da8-b5670bf3d6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:26.385689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4881f25-d0b3-477c-a63c-978d7d5713f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:26.385803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad7b6e-1ed1-439b-8da8-b5670bf3d6b9 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.386138 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad7b6e-1ed1-439b-8da8-b5670bf3d6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.386238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4881f25-d0b3-477c-a63c-978d7d5713f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:26.386578 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad7b6e-1ed1-439b-8da8-b5670bf3d6b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.387829 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad7b6e-1ed1-439b-8da8-b5670bf3d6b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:26.388567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4881f25-d0b3-477c-a63c-978d7d5713f2 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:26.390120 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad7b6e-1ed1-439b-8da8-b5670bf3d6b9 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:26.411650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 418/810] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:26.412870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4881f25-d0b3-477c-a63c-978d7d5713f2 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:26.432008 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 393/811] 10.222.0.60 () {60 vars in 1144 bytes} [Thu Jul 21 20:39:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:26.433490 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cb649-9edb-4b26-ab04-973f06c8d55d None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.433928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cb649-9edb-4b26-ab04-973f06c8d55d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.434222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cb649-9edb-4b26-ab04-973f06c8d55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.450790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:26.459367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-545cb649-9edb-4b26-ab04-973f06c8d55d None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:26.460701 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-545cb649-9edb-4b26-ab04-973f06c8d55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:26.532488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:26.543279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4881f25-d0b3-477c-a63c-978d7d5713f2 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:26.557435 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 394/812] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:26.558321 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545cb649-9edb-4b26-ab04-973f06c8d55d None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:26.594574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256421a-a0ea-40c8-a339-22e7f1ac3dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:26.594809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256421a-a0ea-40c8-a339-22e7f1ac3dcf None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.594977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256421a-a0ea-40c8-a339-22e7f1ac3dcf None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.595696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256421a-a0ea-40c8-a339-22e7f1ac3dcf None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.596047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256421a-a0ea-40c8-a339-22e7f1ac3dcf None admin] RBAC: Adding query filter params (name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:26.596356 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256421a-a0ea-40c8-a339-22e7f1ac3dcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:26.597964 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256421a-a0ea-40c8-a339-22e7f1ac3dcf None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:26.609110 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 419/813] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:39:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:26.609824 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256421a-a0ea-40c8-a339-22e7f1ac3dcf None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:26.700644 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545cb649-9edb-4b26-ab04-973f06c8d55d None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:26.728490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:26.737787 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34105cf2-7b47-4da0-a8df-9929e4480e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:26.737988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256421a-a0ea-40c8-a339-22e7f1ac3dcf None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:26.738583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34105cf2-7b47-4da0-a8df-9929e4480e09 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.738863 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34105cf2-7b47-4da0-a8df-9929e4480e09 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.739853 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34105cf2-7b47-4da0-a8df-9929e4480e09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.749047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34105cf2-7b47-4da0-a8df-9929e4480e09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:26.750510 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34105cf2-7b47-4da0-a8df-9929e4480e09 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:26.764923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-34105cf2-7b47-4da0-a8df-9929e4480e09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:26.764923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:26.764923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:26.764923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:26.764923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:26.764923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:26.764923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:26.764923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:26.764923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:26.764923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:26.764923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:26.764923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:26.764923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:26.764923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:26.764923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:26.765854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:26.766741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:26.766741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:26.766741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:26.766741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:26.766741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:26.766741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:26.766741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:26.766741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:26.766741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 395/814] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:26.767259 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34105cf2-7b47-4da0-a8df-9929e4480e09 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:26.785389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782d0ae-1189-4c32-ac89-8f4efb5c48c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:26.786962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782d0ae-1189-4c32-ac89-8f4efb5c48c9 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.787456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782d0ae-1189-4c32-ac89-8f4efb5c48c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.788072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782d0ae-1189-4c32-ac89-8f4efb5c48c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.796155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782d0ae-1189-4c32-ac89-8f4efb5c48c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:26.797608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782d0ae-1189-4c32-ac89-8f4efb5c48c9 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:26.825163 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:26.890770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34105cf2-7b47-4da0-a8df-9929e4480e09 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:26.931911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-e782d0ae-1189-4c32-ac89-8f4efb5c48c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:26.931911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:26.931911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:26.931911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:26.931911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:26.931911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:26.931911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:26.931911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:26.931911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:26.931911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:26.931911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:26.931911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:26.931911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:26.931911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:26.931911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:26.933041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:26.933931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:26.933931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:26.933931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:26.933931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:26.933931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:26.933931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:26.933931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:26.933931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:26.933931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 420/815] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:26.933931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782d0ae-1189-4c32-ac89-8f4efb5c48c9 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:26.950528 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09154d-6aee-44e5-8008-4d3f876f5041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:26.951397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09154d-6aee-44e5-8008-4d3f876f5041 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:26.951633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09154d-6aee-44e5-8008-4d3f876f5041 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:26.951955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09154d-6aee-44e5-8008-4d3f876f5041 None admin] PATH_INFO: `/v3/services/network` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:26.952650 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca09154d-6aee-44e5-8008-4d3f876f5041 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:26.954024 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca09154d-6aee-44e5-8008-4d3f876f5041 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-ca09154d-6aee-44e5-8008-4d3f876f5041 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:26.967777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:26.969376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:26.970209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:26.970209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:26.970209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:26.970209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:26.970209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:26.970209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:26.970209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:26.970209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:26.970209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:26.970209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:26.970209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:26.970209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:26.970209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 20:39:26.970209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:26.970209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 396/816] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:26 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:26.971006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09154d-6aee-44e5-8008-4d3f876f5041 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:27.066396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782d0ae-1189-4c32-ac89-8f4efb5c48c9 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:27.088261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09154d-6aee-44e5-8008-4d3f876f5041 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:27.103124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:27.111186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35671b-87d2-47c0-98b7-e8b7e16a35c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:27.112024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35671b-87d2-47c0-98b7-e8b7e16a35c3 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:27.112382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35671b-87d2-47c0-98b7-e8b7e16a35c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:27.112706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35671b-87d2-47c0-98b7-e8b7e16a35c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:27.125999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35671b-87d2-47c0-98b7-e8b7e16a35c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:27.127462 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35671b-87d2-47c0-98b7-e8b7e16a35c3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:27.136956 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7db40f-f8a8-4025-8e0b-73664c1577fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:27.137750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7db40f-f8a8-4025-8e0b-73664c1577fb None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:27.138036 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7db40f-f8a8-4025-8e0b-73664c1577fb None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:27.138315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7db40f-f8a8-4025-8e0b-73664c1577fb None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:27.139117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7db40f-f8a8-4025-8e0b-73664c1577fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:27.139425 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7db40f-f8a8-4025-8e0b-73664c1577fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:27.140895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7db40f-f8a8-4025-8e0b-73664c1577fb None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:27.149760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-2c35671b-87d2-47c0-98b7-e8b7e16a35c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:27.149760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:27.149760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:27.149760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:27.149760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:27.149760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:27.149760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:27.149760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:27.149760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:27.149760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:27.149760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:27.149760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:27.149760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:27.149760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:27.149760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:27.150894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:27.152030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:27.152030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:27.152030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:27.152030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:27.153633 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 421/817] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:27.154633 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35671b-87d2-47c0-98b7-e8b7e16a35c3 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:27.162092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:27.168311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 397/818] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:27.169818 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7db40f-f8a8-4025-8e0b-73664c1577fb None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:27.274492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35671b-87d2-47c0-98b7-e8b7e16a35c3 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:27.287595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7db40f-f8a8-4025-8e0b-73664c1577fb None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:27.321317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a08a1-b4bc-4b7d-a506-ec7fbcd4bee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:27.322752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a08a1-b4bc-4b7d-a506-ec7fbcd4bee3 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:27.323038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a08a1-b4bc-4b7d-a506-ec7fbcd4bee3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:27.323313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a08a1-b4bc-4b7d-a506-ec7fbcd4bee3 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:27.324176 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a08a1-b4bc-4b7d-a506-ec7fbcd4bee3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:27.324483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a08a1-b4bc-4b7d-a506-ec7fbcd4bee3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:27.325924 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a08a1-b4bc-4b7d-a506-ec7fbcd4bee3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:27.334898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef556376-cd49-4fed-b0f4-f6a28d3cbfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:27.335582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef556376-cd49-4fed-b0f4-f6a28d3cbfbe None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:27.335906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef556376-cd49-4fed-b0f4-f6a28d3cbfbe None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:27.336181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef556376-cd49-4fed-b0f4-f6a28d3cbfbe None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:27.336893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef556376-cd49-4fed-b0f4-f6a28d3cbfbe None admin] RBAC: Adding query filter params (name=network) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:27.337216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef556376-cd49-4fed-b0f4-f6a28d3cbfbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:27.338566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef556376-cd49-4fed-b0f4-f6a28d3cbfbe None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:27.344068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 422/819] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:27.344068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a08a1-b4bc-4b7d-a506-ec7fbcd4bee3 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:27.345319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 398/820] 10.222.0.60 () {56 vars in 1112 bytes} [Thu Jul 21 20:39:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:27.350190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef556376-cd49-4fed-b0f4-f6a28d3cbfbe None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:27.390262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:27.420729 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:27.467621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a08a1-b4bc-4b7d-a506-ec7fbcd4bee3 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:27.468910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef556376-cd49-4fed-b0f4-f6a28d3cbfbe None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:27.507029 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781ff6a-8c53-43df-8e04-222e3354b219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:27.507690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781ff6a-8c53-43df-8e04-222e3354b219 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:27.508000 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781ff6a-8c53-43df-8e04-222e3354b219 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:27.508272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781ff6a-8c53-43df-8e04-222e3354b219 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:27.509104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781ff6a-8c53-43df-8e04-222e3354b219 None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=77812b356c974d2b80102217956169db) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:27.509338 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72b88f-77a1-4264-871a-4038fa1a9c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:27.509465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781ff6a-8c53-43df-8e04-222e3354b219 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:27.510001 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72b88f-77a1-4264-871a-4038fa1a9c74 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:27.510282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72b88f-77a1-4264-871a-4038fa1a9c74 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:27.510554 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72b88f-77a1-4264-871a-4038fa1a9c74 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:27.510825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781ff6a-8c53-43df-8e04-222e3354b219 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:27.511267 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72b88f-77a1-4264-871a-4038fa1a9c74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:27.511567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72b88f-77a1-4264-871a-4038fa1a9c74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:27.513289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72b88f-77a1-4264-871a-4038fa1a9c74 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:27.526536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 399/821] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:39:27 2022] GET /identity/v3/role_assignments?user.id=77812b356c974d2b80102217956169db&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:27.526536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781ff6a-8c53-43df-8e04-222e3354b219 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:27.533592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 423/822] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:27.536428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72b88f-77a1-4264-871a-4038fa1a9c74 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:27.697696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72b88f-77a1-4264-871a-4038fa1a9c74 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:27.729493 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:27.741791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52926345-a162-428a-bd2d-d87a7e80ea17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:27.742474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52926345-a162-428a-bd2d-d87a7e80ea17 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:27.742757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52926345-a162-428a-bd2d-d87a7e80ea17 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:27.743041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52926345-a162-428a-bd2d-d87a7e80ea17 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:27.743787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52926345-a162-428a-bd2d-d87a7e80ea17 None admin] RBAC: Adding query filter params (type=network) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:27.744102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52926345-a162-428a-bd2d-d87a7e80ea17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:27.745489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52926345-a162-428a-bd2d-d87a7e80ea17 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:27.751317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 424/823] 10.222.0.60 () {56 vars in 1112 bytes} [Thu Jul 21 20:39:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:27.752022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52926345-a162-428a-bd2d-d87a7e80ea17 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:27.780572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:27.786887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781ff6a-8c53-43df-8e04-222e3354b219 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:27.834797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d1aeb-a844-4731-9c07-b6662404f5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:27.835686 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d1aeb-a844-4731-9c07-b6662404f5f5 None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:27.835989 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d1aeb-a844-4731-9c07-b6662404f5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:27.836275 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d1aeb-a844-4731-9c07-b6662404f5f5 None admin] PATH_INFO: `/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/833a474f828f4349bea0d1649461c77e/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:27.838262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d1aeb-a844-4731-9c07-b6662404f5f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc61fe15894891a9054911321f4ac3, user_id=833a474f828f4349bea0d1649461c77e, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:27.839727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d1aeb-a844-4731-9c07-b6662404f5f5 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:27.858291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 400/824] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:39:27 2022] PUT /identity/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/833a474f828f4349bea0d1649461c77e/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:27.859511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d1aeb-a844-4731-9c07-b6662404f5f5 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:27.900120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52926345-a162-428a-bd2d-d87a7e80ea17 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:27.957042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d175b8a-2b03-472f-b717-6df6ed3dfcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:27.960271 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d175b8a-2b03-472f-b717-6df6ed3dfcb8 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:27.960617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d175b8a-2b03-472f-b717-6df6ed3dfcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:27.960894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d175b8a-2b03-472f-b717-6df6ed3dfcb8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:27.976861 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d175b8a-2b03-472f-b717-6df6ed3dfcb8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:27.978480 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d175b8a-2b03-472f-b717-6df6ed3dfcb8 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-9d175b8a-2b03-472f-b717-6df6ed3dfcb8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:28.018123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:28.019158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:28.020149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:28.020149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:28.020149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:28.020149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:28.020149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:28.020149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:28.020149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:28.020149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:28.020149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:39:28.020149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:28.027489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 425/825] 10.222.0.60 () {56 vars in 1117 bytes} [Thu Jul 21 20:39:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:28.033505 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d175b8a-2b03-472f-b717-6df6ed3dfcb8 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:28.058532 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d1aeb-a844-4731-9c07-b6662404f5f5 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:28.105330 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93bd35-576f-4946-83ca-8607295f1485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:28.106683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93bd35-576f-4946-83ca-8607295f1485 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:28.106972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93bd35-576f-4946-83ca-8607295f1485 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:28.107250 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93bd35-576f-4946-83ca-8607295f1485 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:28.108046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93bd35-576f-4946-83ca-8607295f1485 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:28.109451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93bd35-576f-4946-83ca-8607295f1485 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:28.121173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:28.138841 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 401/826] 10.222.0.60 () {60 vars in 1144 bytes} [Thu Jul 21 20:39:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:28.196426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d175b8a-2b03-472f-b717-6df6ed3dfcb8 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:28.267005 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d70fe-78c1-435c-b66a-fbb9c2fc8b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:28.267736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d70fe-78c1-435c-b66a-fbb9c2fc8b5a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:28.268025 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d70fe-78c1-435c-b66a-fbb9c2fc8b5a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:28.268306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d70fe-78c1-435c-b66a-fbb9c2fc8b5a None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:28.269179 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d70fe-78c1-435c-b66a-fbb9c2fc8b5a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:28.269490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d70fe-78c1-435c-b66a-fbb9c2fc8b5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:28.271294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d70fe-78c1-435c-b66a-fbb9c2fc8b5a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:28.295522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 426/827] 10.222.0.60 () {56 vars in 1149 bytes} [Thu Jul 21 20:39:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:28.300743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93bd35-576f-4946-83ca-8607295f1485 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:28.334003 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:28.431614 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93bd35-576f-4946-83ca-8607295f1485 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:28.474642 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5111d-043a-450c-ae75-9986639cd631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:28.475364 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5111d-043a-450c-ae75-9986639cd631 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:28.475649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5111d-043a-450c-ae75-9986639cd631 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:28.475649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5111d-043a-450c-ae75-9986639cd631 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:28.493799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5111d-043a-450c-ae75-9986639cd631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:28.495301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5111d-043a-450c-ae75-9986639cd631 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:28.518238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-33a5111d-043a-450c-ae75-9986639cd631 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:28.518238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:28.518238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:28.518238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:28.518238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:28.518238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:28.518238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:28.518238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:28.518238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:28.518238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:28.518238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:28.518238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:28.518238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:28.518238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:28.518238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:28.519231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:28.520142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:28.520142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:28.520142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:28.520142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:28.520142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 402/828] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:28.527162 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d70fe-78c1-435c-b66a-fbb9c2fc8b5a None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:28.664154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d70fe-78c1-435c-b66a-fbb9c2fc8b5a None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:28.666169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:28.712245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518768f-af33-4e6c-940c-7359d061da9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:28.713282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518768f-af33-4e6c-940c-7359d061da9e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:28.713762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518768f-af33-4e6c-940c-7359d061da9e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:28.714266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518768f-af33-4e6c-940c-7359d061da9e None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:28.715191 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518768f-af33-4e6c-940c-7359d061da9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:28.715868 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518768f-af33-4e6c-940c-7359d061da9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:28.717538 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518768f-af33-4e6c-940c-7359d061da9e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:28.730906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 427/829] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:28.735378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d64cf-59e4-4d04-a85f-caaa5dd2ccf9 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:28.735378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d64cf-59e4-4d04-a85f-caaa5dd2ccf9 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:28.735378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d64cf-59e4-4d04-a85f-caaa5dd2ccf9 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:28.735780 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 403/830] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:28.736766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518768f-af33-4e6c-940c-7359d061da9e None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:28.740817 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a2741-b33d-4de3-88b9-1a4cbdf2c671 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:28.741101 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a2741-b33d-4de3-88b9-1a4cbdf2c671 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:28.741379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a2741-b33d-4de3-88b9-1a4cbdf2c671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:28.760062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-1a8a2741-b33d-4de3-88b9-1a4cbdf2c671 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:28.761287 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1a8a2741-b33d-4de3-88b9-1a4cbdf2c671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:28.845219 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:28.859011 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 404/831] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:28.864883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518768f-af33-4e6c-940c-7359d061da9e None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:28.885473 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a2741-b33d-4de3-88b9-1a4cbdf2c671 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:28.915154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace372cc-6164-4eec-8cce-e97534a20300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:28.915154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace372cc-6164-4eec-8cce-e97534a20300 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:28.915154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace372cc-6164-4eec-8cce-e97534a20300 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:28.915154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace372cc-6164-4eec-8cce-e97534a20300 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:28.924493 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace372cc-6164-4eec-8cce-e97534a20300 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:28.927044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace372cc-6164-4eec-8cce-e97534a20300 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:28.945719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-ace372cc-6164-4eec-8cce-e97534a20300 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:28.945719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:28.945719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:28.945719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:28.945719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:28.945719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:28.945719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:28.945719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:28.945719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:28.945719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:28.945719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:28.945719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:28.945719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:28.945719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:28.945719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:28.946941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:28.948062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:28.948062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:28.948062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:28.948062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:28.948062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:28.948062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:28.948062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:28.948062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:28.951812 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 428/832] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:28.955034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace372cc-6164-4eec-8cce-e97534a20300 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:28.991745 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:29.023749 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a2741-b33d-4de3-88b9-1a4cbdf2c671 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:29.085708 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da7076-9c0c-4c90-b68d-48268bf94938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:29.086823 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da7076-9c0c-4c90-b68d-48268bf94938 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:29.087227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da7076-9c0c-4c90-b68d-48268bf94938 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:29.087615 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da7076-9c0c-4c90-b68d-48268bf94938 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:29.088396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da7076-9c0c-4c90-b68d-48268bf94938 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:29.089929 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da7076-9c0c-4c90-b68d-48268bf94938 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-a1da7076-9c0c-4c90-b68d-48268bf94938 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:29.103496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:29.105028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:29.106040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:29.106040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:29.106040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:29.106040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:29.106040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:29.106040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:29.106040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:29.106040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:29.106040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:29.106040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:29.106040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:29.106040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:29.106040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:39:29.106040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:29.107412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 405/833] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:39:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:29.109443 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace372cc-6164-4eec-8cce-e97534a20300 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:29.113214 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da7076-9c0c-4c90-b68d-48268bf94938 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:29.157743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ec92c-d24a-4bb2-b779-c3ebf3a6ddc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:29.158994 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ec92c-d24a-4bb2-b779-c3ebf3a6ddc1 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:29.159423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ec92c-d24a-4bb2-b779-c3ebf3a6ddc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:29.159865 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ec92c-d24a-4bb2-b779-c3ebf3a6ddc1 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:29.161999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ec92c-d24a-4bb2-b779-c3ebf3a6ddc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:29.162880 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ec92c-d24a-4bb2-b779-c3ebf3a6ddc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:29.164822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ec92c-d24a-4bb2-b779-c3ebf3a6ddc1 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:29.180735 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:29.185700 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 429/834] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:29.191011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ec92c-d24a-4bb2-b779-c3ebf3a6ddc1 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:29.238209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da7076-9c0c-4c90-b68d-48268bf94938 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:29.276345 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978048f-c712-440f-976e-e338d1695d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:29.277084 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978048f-c712-440f-976e-e338d1695d6d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:29.277369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978048f-c712-440f-976e-e338d1695d6d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:29.277700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978048f-c712-440f-976e-e338d1695d6d None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:29.278726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978048f-c712-440f-976e-e338d1695d6d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:29.278726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978048f-c712-440f-976e-e338d1695d6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:29.280129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978048f-c712-440f-976e-e338d1695d6d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:29.291905 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 406/835] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:39:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:29.295332 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978048f-c712-440f-976e-e338d1695d6d None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:29.299522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:29.330050 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ec92c-d24a-4bb2-b779-c3ebf3a6ddc1 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:29.380213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d926e-49ce-4b22-afc5-1ff6b6356645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:29.381968 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d926e-49ce-4b22-afc5-1ff6b6356645 None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:29.382402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d926e-49ce-4b22-afc5-1ff6b6356645 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:29.383125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d926e-49ce-4b22-afc5-1ff6b6356645 None admin] PATH_INFO: `/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/f0beb65e5dc7469eb17a98dae1f09a0e/roles/05f3681d0a644607934e281f75ecd675` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:29.402684 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d926e-49ce-4b22-afc5-1ff6b6356645 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc61fe15894891a9054911321f4ac3, user_id=f0beb65e5dc7469eb17a98dae1f09a0e, role_id=05f3681d0a644607934e281f75ecd675)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:29.404829 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d926e-49ce-4b22-afc5-1ff6b6356645 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:29.423030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 430/836] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:39:29 2022] PUT /identity/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/f0beb65e5dc7469eb17a98dae1f09a0e/roles/05f3681d0a644607934e281f75ecd675 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:29.434493 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978048f-c712-440f-976e-e338d1695d6d None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:29.468197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:29.481358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e8466-ed76-4b94-ac84-67d1f2fc3bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:29.482360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e8466-ed76-4b94-ac84-67d1f2fc3bac None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:29.482800 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e8466-ed76-4b94-ac84-67d1f2fc3bac None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:29.483312 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e8466-ed76-4b94-ac84-67d1f2fc3bac None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:29.484245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e8466-ed76-4b94-ac84-67d1f2fc3bac None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:29.484710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e8466-ed76-4b94-ac84-67d1f2fc3bac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:29.486306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e8466-ed76-4b94-ac84-67d1f2fc3bac None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:29.493597 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 407/837] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:39:29 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:30.017060 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a069d-8a3b-4427-b916-1bfabfe4306f None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:30.017403 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a069d-8a3b-4427-b916-1bfabfe4306f None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:30.017707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a069d-8a3b-4427-b916-1bfabfe4306f None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:30.019341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 431/838] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:30.025511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4aea1-ffc9-4d94-892e-bdd8d30df27e None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:30.025802 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4aea1-ffc9-4d94-892e-bdd8d30df27e None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:30.026079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4aea1-ffc9-4d94-892e-bdd8d30df27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:30.044427 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-83f4aea1-ffc9-4d94-892e-bdd8d30df27e None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:30.045637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-83f4aea1-ffc9-4d94-892e-bdd8d30df27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:30.121972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 408/839] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:30.144037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a069d-8a3b-4427-b916-1bfabfe4306f None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:30.263284 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a069d-8a3b-4427-b916-1bfabfe4306f None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:30.265819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090fc94-be62-4cd0-859c-f3204abe5e0a None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:30.266181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090fc94-be62-4cd0-859c-f3204abe5e0a None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:30.266532 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090fc94-be62-4cd0-859c-f3204abe5e0a None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:30.268373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 409/840] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:39:30.274744 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f25f97-07ed-4821-a2b0-13ee40cd8a91 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:30.275302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f25f97-07ed-4821-a2b0-13ee40cd8a91 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:30.275797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f25f97-07ed-4821-a2b0-13ee40cd8a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:30.300376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-04f25f97-07ed-4821-a2b0-13ee40cd8a91 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:30.303525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-04f25f97-07ed-4821-a2b0-13ee40cd8a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:30.315648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:30.322630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82133a66-971e-4e97-ad2c-52a6910d9026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:30.323821 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82133a66-971e-4e97-ad2c-52a6910d9026 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:30.324288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82133a66-971e-4e97-ad2c-52a6910d9026 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:30.324834 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82133a66-971e-4e97-ad2c-52a6910d9026 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:30.325694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82133a66-971e-4e97-ad2c-52a6910d9026 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:30.327410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82133a66-971e-4e97-ad2c-52a6910d9026 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-82133a66-971e-4e97-ad2c-52a6910d9026 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:30.342068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:30.343066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:30.343932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:30.343932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:30.343932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:30.343932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:30.343932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:30.343932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:30.343932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:30.343932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:30.343932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:30.343932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:30.343932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:30.343932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:30.343932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:30.343932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:39:30.343932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:30.347008 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 432/841] 10.222.0.60 () {56 vars in 1132 bytes} [Thu Jul 21 20:39:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:30.354491 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82133a66-971e-4e97-ad2c-52a6910d9026 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:30.411473 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 410/842] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:30.437980 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f25f97-07ed-4821-a2b0-13ee40cd8a91 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:30.477460 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82133a66-971e-4e97-ad2c-52a6910d9026 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:30.513064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:30.519255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2d2fa-1ddf-42d3-bae0-345bb2bed63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:30.520676 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2d2fa-1ddf-42d3-bae0-345bb2bed63b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:30.521440 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2d2fa-1ddf-42d3-bae0-345bb2bed63b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:30.521973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2d2fa-1ddf-42d3-bae0-345bb2bed63b None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:30.522827 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2d2fa-1ddf-42d3-bae0-345bb2bed63b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:30.523289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2d2fa-1ddf-42d3-bae0-345bb2bed63b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:30.525308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2d2fa-1ddf-42d3-bae0-345bb2bed63b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:30.531002 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 433/843] 10.222.0.60 () {56 vars in 1126 bytes} [Thu Jul 21 20:39:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:30.538234 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2d2fa-1ddf-42d3-bae0-345bb2bed63b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:30.574592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f25f97-07ed-4821-a2b0-13ee40cd8a91 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:30.620387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2aef9-737f-48d0-addb-4effc6653a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:30.621447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2aef9-737f-48d0-addb-4effc6653a77 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:30.621879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2aef9-737f-48d0-addb-4effc6653a77 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:30.622326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2aef9-737f-48d0-addb-4effc6653a77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:30.635029 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:30.635882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2aef9-737f-48d0-addb-4effc6653a77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-2aa2aef9-737f-48d0-addb-4effc6653a77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:30.646521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:30.647883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:30.648933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:30.648933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:30.648933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:30.648933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:30.648933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:30.648933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:30.648933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:30.648933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:30.648933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:30.648933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:30.648933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:39:30.648933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:30.656327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 411/844] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:39:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:30.657491 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2aef9-737f-48d0-addb-4effc6653a77 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:30.689166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2d2fa-1ddf-42d3-bae0-345bb2bed63b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:30.738919 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319d0ad-cd00-400f-bfef-d3cd5ba605c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:30.740022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319d0ad-cd00-400f-bfef-d3cd5ba605c4 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:30.740411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319d0ad-cd00-400f-bfef-d3cd5ba605c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:30.740910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319d0ad-cd00-400f-bfef-d3cd5ba605c4 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:30.742590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319d0ad-cd00-400f-bfef-d3cd5ba605c4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:30.742985 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319d0ad-cd00-400f-bfef-d3cd5ba605c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:30.745098 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319d0ad-cd00-400f-bfef-d3cd5ba605c4 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:30.751487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 434/845] 10.222.0.60 () {56 vars in 1126 bytes} [Thu Jul 21 20:39:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:30.781607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2aef9-737f-48d0-addb-4effc6653a77 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:30.793826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:30.823220 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9adec6-2b7b-4143-ad02-ee7f1377d5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:30.824052 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9adec6-2b7b-4143-ad02-ee7f1377d5e6 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:30.824338 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9adec6-2b7b-4143-ad02-ee7f1377d5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:30.824745 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9adec6-2b7b-4143-ad02-ee7f1377d5e6 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:30.825525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9adec6-2b7b-4143-ad02-ee7f1377d5e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:30.825831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9adec6-2b7b-4143-ad02-ee7f1377d5e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:30.827253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9adec6-2b7b-4143-ad02-ee7f1377d5e6 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:30.839147 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 412/846] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:39:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:30.844889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319d0ad-cd00-400f-bfef-d3cd5ba605c4 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:30.967705 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319d0ad-cd00-400f-bfef-d3cd5ba605c4 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:30.996310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:31.004057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca386ac0-5fe6-485f-8da0-9d6f37deafda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:31.004978 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca386ac0-5fe6-485f-8da0-9d6f37deafda None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:31.005682 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca386ac0-5fe6-485f-8da0-9d6f37deafda None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:31.005967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca386ac0-5fe6-485f-8da0-9d6f37deafda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:31.017332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca386ac0-5fe6-485f-8da0-9d6f37deafda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:31.018783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca386ac0-5fe6-485f-8da0-9d6f37deafda None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:31.037186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-ca386ac0-5fe6-485f-8da0-9d6f37deafda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:31.037186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:31.037186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:31.037186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:31.037186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:31.037186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:31.037186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:31.037186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:31.037186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:31.037186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:31.037186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:31.037186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:31.037186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:31.037186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:31.037186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:31.038084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:31.038942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:31.038942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:31.038942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:31.038942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:31.038942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 435/847] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:31.045414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9adec6-2b7b-4143-ad02-ee7f1377d5e6 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:31.157150 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9adec6-2b7b-4143-ad02-ee7f1377d5e6 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:31.195029 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56ad51-1382-40d8-b051-9eff3e5ff399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:31.195706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56ad51-1382-40d8-b051-9eff3e5ff399 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:31.196017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56ad51-1382-40d8-b051-9eff3e5ff399 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:31.196292 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56ad51-1382-40d8-b051-9eff3e5ff399 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:31.197022 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56ad51-1382-40d8-b051-9eff3e5ff399 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:31.197325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56ad51-1382-40d8-b051-9eff3e5ff399 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:31.198705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56ad51-1382-40d8-b051-9eff3e5ff399 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:31.212115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 413/848] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:31.217694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca386ac0-5fe6-485f-8da0-9d6f37deafda None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:31.350346 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca386ac0-5fe6-485f-8da0-9d6f37deafda None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:31.391923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c69fac-768f-4404-84eb-53e81c8164bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:31.392765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c69fac-768f-4404-84eb-53e81c8164bc None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:31.393055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c69fac-768f-4404-84eb-53e81c8164bc None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:31.393337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c69fac-768f-4404-84eb-53e81c8164bc None admin] PATH_INFO: `/v3/users/swift` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:31.402616 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c69fac-768f-4404-84eb-53e81c8164bc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:31.404107 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c69fac-768f-4404-84eb-53e81c8164bc None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-01c69fac-768f-4404-84eb-53e81c8164bc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:31.421037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:31.422013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:31.422930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:31.422930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:31.422930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:31.422930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:31.422930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:31.422930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:31.422930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:31.422930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:31.422930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 20:39:31.422930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:31.422930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 436/849] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:39:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:31.431326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56ad51-1382-40d8-b051-9eff3e5ff399 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:31.474657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:31.569177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56ad51-1382-40d8-b051-9eff3e5ff399 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:31.622232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03964414-633e-409d-b70c-af2321fd983a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:31.622840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03964414-633e-409d-b70c-af2321fd983a None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:31.623128 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03964414-633e-409d-b70c-af2321fd983a None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:31.623408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03964414-633e-409d-b70c-af2321fd983a None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:31.624589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03964414-633e-409d-b70c-af2321fd983a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:31.625163 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03964414-633e-409d-b70c-af2321fd983a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:31.626784 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03964414-633e-409d-b70c-af2321fd983a None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:31.656476 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 414/850] 10.222.0.60 () {56 vars in 1135 bytes} [Thu Jul 21 20:39:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:31.663252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c69fac-768f-4404-84eb-53e81c8164bc None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:31.712141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:31.725507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda77241-1487-4239-9124-b21f7224831c None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:31.725837 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda77241-1487-4239-9124-b21f7224831c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:31.726081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda77241-1487-4239-9124-b21f7224831c None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:31.727650 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 415/851] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:31.739553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951b090-8a0a-4027-9db7-ebeff414044c None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:31.740535 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951b090-8a0a-4027-9db7-ebeff414044c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:31.741020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951b090-8a0a-4027-9db7-ebeff414044c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:31.761784 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c951b090-8a0a-4027-9db7-ebeff414044c None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:31.763556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c951b090-8a0a-4027-9db7-ebeff414044c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:31.809440 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c69fac-768f-4404-84eb-53e81c8164bc None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:31.853492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72874de-a4ac-41e2-80da-1da67cb1f865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:31.854746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72874de-a4ac-41e2-80da-1da67cb1f865 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:31.859426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72874de-a4ac-41e2-80da-1da67cb1f865 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:31.859837 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72874de-a4ac-41e2-80da-1da67cb1f865 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:31.870108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 416/852] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:31.871285 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672fb11-8a34-49f5-8b0a-9de2ad51f4a9 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:31.871646 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672fb11-8a34-49f5-8b0a-9de2ad51f4a9 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:31.871973 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672fb11-8a34-49f5-8b0a-9de2ad51f4a9 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:31.874033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 417/853] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:31.879795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b66727-aa0d-4ec8-bbd7-2e89b9b46586 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:31.880273 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b66727-aa0d-4ec8-bbd7-2e89b9b46586 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:31.880557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b66727-aa0d-4ec8-bbd7-2e89b9b46586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:31.901491 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d0b66727-aa0d-4ec8-bbd7-2e89b9b46586 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:31.902822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d0b66727-aa0d-4ec8-bbd7-2e89b9b46586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:31.918070 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:31.953452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72874de-a4ac-41e2-80da-1da67cb1f865 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:31.955187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72874de-a4ac-41e2-80da-1da67cb1f865 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:31.997553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:32.001731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 418/854] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:32.011939 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b66727-aa0d-4ec8-bbd7-2e89b9b46586 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:32.090920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 437/855] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:39:31 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:39:32.090920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72874de-a4ac-41e2-80da-1da67cb1f865 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:32.173084 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b66727-aa0d-4ec8-bbd7-2e89b9b46586 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:32.215477 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72874de-a4ac-41e2-80da-1da67cb1f865 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:32.220501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba63ada-2d13-4c37-b1bf-13fc128c07d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:32.221208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba63ada-2d13-4c37-b1bf-13fc128c07d8 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:32.221477 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba63ada-2d13-4c37-b1bf-13fc128c07d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:32.223118 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba63ada-2d13-4c37-b1bf-13fc128c07d8 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:32.225673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba63ada-2d13-4c37-b1bf-13fc128c07d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:32.228023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba63ada-2d13-4c37-b1bf-13fc128c07d8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:32.245522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 419/856] 10.222.0.60 () {60 vars in 1141 bytes} [Thu Jul 21 20:39:32 2022] POST /identity/v3/services => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:32.246656 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba63ada-2d13-4c37-b1bf-13fc128c07d8 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:32.270005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:32.274293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:32.282677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face3536-805a-4f9c-b533-d282e5f964f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:32.285220 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face3536-805a-4f9c-b533-d282e5f964f4 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:32.286913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face3536-805a-4f9c-b533-d282e5f964f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:32.287761 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face3536-805a-4f9c-b533-d282e5f964f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:32.312815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face3536-805a-4f9c-b533-d282e5f964f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-face3536-805a-4f9c-b533-d282e5f964f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:32.322617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:32.323873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:32.324845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:32.324845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:32.324845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:32.324845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:32.324845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:32.324845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:32.324845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:32.324845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:32.324845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:32.324845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:32.324845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:39:32.324845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:32.331934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 438/857] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:39:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:32.333994 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face3536-805a-4f9c-b533-d282e5f964f4 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:32.421443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba63ada-2d13-4c37-b1bf-13fc128c07d8 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:32.481011 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f1a0f-1e88-43f1-9cb8-d29c9044ee39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:32.481754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f1a0f-1e88-43f1-9cb8-d29c9044ee39 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:32.482028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f1a0f-1e88-43f1-9cb8-d29c9044ee39 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:32.482291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f1a0f-1e88-43f1-9cb8-d29c9044ee39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:32.496428 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f1a0f-1e88-43f1-9cb8-d29c9044ee39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:32.499123 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f1a0f-1e88-43f1-9cb8-d29c9044ee39 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:32.505583 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face3536-805a-4f9c-b533-d282e5f964f4 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:32.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-632f1a0f-1e88-43f1-9cb8-d29c9044ee39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:32.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:32.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:32.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:32.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:32.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:32.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:32.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:32.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:32.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:32.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:32.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:32.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:32.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:32.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:32.522412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:32.523403 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:32.523403 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:32.523403 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:32.523403 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:32.525020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 420/858] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:32.531926 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f1a0f-1e88-43f1-9cb8-d29c9044ee39 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:32.562325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a38449-5dad-483a-aa79-ca40e49cfd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:32.563418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a38449-5dad-483a-aa79-ca40e49cfd48 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:32.564224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a38449-5dad-483a-aa79-ca40e49cfd48 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:32.564657 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a38449-5dad-483a-aa79-ca40e49cfd48 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:32.565839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a38449-5dad-483a-aa79-ca40e49cfd48 None admin] RBAC: Adding query filter params (name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:32.566454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a38449-5dad-483a-aa79-ca40e49cfd48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:32.568735 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a38449-5dad-483a-aa79-ca40e49cfd48 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:32.584120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 439/859] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:39:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:32.593619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a38449-5dad-483a-aa79-ca40e49cfd48 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:32.609344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:32.659276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:32.675613 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f1a0f-1e88-43f1-9cb8-d29c9044ee39 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:32.726351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a38449-5dad-483a-aa79-ca40e49cfd48 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:32.736921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e414f8-f950-4152-a772-d559b304dd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:32.738322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e414f8-f950-4152-a772-d559b304dd4d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:32.738812 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e414f8-f950-4152-a772-d559b304dd4d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:32.739277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e414f8-f950-4152-a772-d559b304dd4d None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:32.740945 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e414f8-f950-4152-a772-d559b304dd4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:32.741418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e414f8-f950-4152-a772-d559b304dd4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:32.743782 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e414f8-f950-4152-a772-d559b304dd4d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:32.767507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 421/860] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:32.769198 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e414f8-f950-4152-a772-d559b304dd4d None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:32.772908 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f8625-b1f7-4e18-af79-0e58d9af6fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:32.773853 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f8625-b1f7-4e18-af79-0e58d9af6fd6 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:32.774260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f8625-b1f7-4e18-af79-0e58d9af6fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:32.774598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f8625-b1f7-4e18-af79-0e58d9af6fd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:32.786988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f8625-b1f7-4e18-af79-0e58d9af6fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:32.788705 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f8625-b1f7-4e18-af79-0e58d9af6fd6 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:32.807753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-b92f8625-b1f7-4e18-af79-0e58d9af6fd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:32.807753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:32.807753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:32.807753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:32.807753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:32.807753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:32.807753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:32.807753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:32.807753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:32.807753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:32.807753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:32.807753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:32.807753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:32.807753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:32.807753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:32.808906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:32.809989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:32.809989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:32.809989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:32.809989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:32.815989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 440/861] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:32.817608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f8625-b1f7-4e18-af79-0e58d9af6fd6 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:32.913320 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e414f8-f950-4152-a772-d559b304dd4d None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:32.936877 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:32.953522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3eed7-9800-4bbc-8ad6-15cf5d867998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:32.954469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3eed7-9800-4bbc-8ad6-15cf5d867998 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:32.954939 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3eed7-9800-4bbc-8ad6-15cf5d867998 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:32.955307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3eed7-9800-4bbc-8ad6-15cf5d867998 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:32.956607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f8625-b1f7-4e18-af79-0e58d9af6fd6 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:32.963128 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3eed7-9800-4bbc-8ad6-15cf5d867998 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:32.964758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3eed7-9800-4bbc-8ad6-15cf5d867998 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:32.966341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:32.980700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-abb3eed7-9800-4bbc-8ad6-15cf5d867998 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:32.980700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:32.980700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:32.980700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:32.980700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:32.980700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:32.980700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:32.980700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:32.980700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:32.980700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:32.980700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:32.980700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:32.980700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:32.980700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:32.980700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:32.982093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:32.983225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:32.983225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:32.983225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:32.983225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:32.983225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:32.983225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:32.983225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:32.983225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:32.990294 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 422/862] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:32.991311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3eed7-9800-4bbc-8ad6-15cf5d867998 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:33.014046 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779cf8c-b1da-4600-bfa4-9bfe5f807132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:33.015118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779cf8c-b1da-4600-bfa4-9bfe5f807132 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:33.015555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779cf8c-b1da-4600-bfa4-9bfe5f807132 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:33.016545 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779cf8c-b1da-4600-bfa4-9bfe5f807132 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:33.018028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779cf8c-b1da-4600-bfa4-9bfe5f807132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:33.018633 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779cf8c-b1da-4600-bfa4-9bfe5f807132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:33.021322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779cf8c-b1da-4600-bfa4-9bfe5f807132 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:33.039704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 441/863] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:33.046025 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779cf8c-b1da-4600-bfa4-9bfe5f807132 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:33.132530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3eed7-9800-4bbc-8ad6-15cf5d867998 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:33.169126 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779cf8c-b1da-4600-bfa4-9bfe5f807132 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:33.181898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f9fdb-b5b5-4909-adb6-a34be5c53d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:33.183003 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f9fdb-b5b5-4909-adb6-a34be5c53d11 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:33.183998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f9fdb-b5b5-4909-adb6-a34be5c53d11 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:33.184473 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f9fdb-b5b5-4909-adb6-a34be5c53d11 None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:33.186293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f9fdb-b5b5-4909-adb6-a34be5c53d11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:33.186815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f9fdb-b5b5-4909-adb6-a34be5c53d11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:33.188317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f9fdb-b5b5-4909-adb6-a34be5c53d11 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:33.206542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 423/864] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:33.206542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce3208-7c43-4346-a928-87bedefb0217 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:33.206542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce3208-7c43-4346-a928-87bedefb0217 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:33.206542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce3208-7c43-4346-a928-87bedefb0217 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:33.206542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 424/865] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:33.212322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce3208-7c43-4346-a928-87bedefb0217 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:33.216783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8187763-1db0-459d-86b2-efa3b55c98cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:33.217591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8187763-1db0-459d-86b2-efa3b55c98cc None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:33.217872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8187763-1db0-459d-86b2-efa3b55c98cc None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:33.218204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8187763-1db0-459d-86b2-efa3b55c98cc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:33.227392 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8187763-1db0-459d-86b2-efa3b55c98cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:33.228866 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8187763-1db0-459d-86b2-efa3b55c98cc None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-c8187763-1db0-459d-86b2-efa3b55c98cc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:33.247308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:33.248539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:33.249511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:33.249511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:33.249511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:33.249511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:33.249511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:33.249511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:33.249511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:33.249511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:33.249511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:39:33.249511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:33.249511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 442/866] 10.222.0.60 () {56 vars in 1117 bytes} [Thu Jul 21 20:39:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:33.250093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d96056-7bff-4771-a496-a5011e0639e6 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:33.250093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d96056-7bff-4771-a496-a5011e0639e6 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:33.250221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d96056-7bff-4771-a496-a5011e0639e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:33.262058 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:33.270740 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d2d96056-7bff-4771-a496-a5011e0639e6 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:33.272084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d2d96056-7bff-4771-a496-a5011e0639e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:33.295203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:33.329767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce3208-7c43-4346-a928-87bedefb0217 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:33.356931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 443/867] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:33.367537 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d96056-7bff-4771-a496-a5011e0639e6 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:33.377059 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa075fd-6e85-4a11-9705-1e3f42bfc30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:33.377968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa075fd-6e85-4a11-9705-1e3f42bfc30c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:33.378432 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa075fd-6e85-4a11-9705-1e3f42bfc30c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:33.378964 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa075fd-6e85-4a11-9705-1e3f42bfc30c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:33.380175 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa075fd-6e85-4a11-9705-1e3f42bfc30c None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=833a474f828f4349bea0d1649461c77e) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:33.380491 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa075fd-6e85-4a11-9705-1e3f42bfc30c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:33.381941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa075fd-6e85-4a11-9705-1e3f42bfc30c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:33.388594 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 425/868] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:39:33 2022] GET /identity/v3/role_assignments?user.id=833a474f828f4349bea0d1649461c77e&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:33.389223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa075fd-6e85-4a11-9705-1e3f42bfc30c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:33.524790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa075fd-6e85-4a11-9705-1e3f42bfc30c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:33.527528 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:33.533585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d96056-7bff-4771-a496-a5011e0639e6 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:33.578952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee99f0b-c691-421f-b1f1-88acd4ebca12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:33.579639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee99f0b-c691-421f-b1f1-88acd4ebca12 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:33.579982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee99f0b-c691-421f-b1f1-88acd4ebca12 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:33.580260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee99f0b-c691-421f-b1f1-88acd4ebca12 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:33.581013 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee99f0b-c691-421f-b1f1-88acd4ebca12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:33.583751 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee99f0b-c691-421f-b1f1-88acd4ebca12 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:33.588809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cda422-7f6f-41b8-9375-ad1f6459f214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:33.589564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cda422-7f6f-41b8-9375-ad1f6459f214 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:33.589746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cda422-7f6f-41b8-9375-ad1f6459f214 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:33.590753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cda422-7f6f-41b8-9375-ad1f6459f214 None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:33.590995 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cda422-7f6f-41b8-9375-ad1f6459f214 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:33.591805 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cda422-7f6f-41b8-9375-ad1f6459f214 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:33.593058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cda422-7f6f-41b8-9375-ad1f6459f214 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:33.604033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 426/869] 10.222.0.60 () {60 vars in 1141 bytes} [Thu Jul 21 20:39:33 2022] POST /identity/v3/services => generated 269 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:33.617470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 444/870] 10.222.0.60 () {56 vars in 1149 bytes} [Thu Jul 21 20:39:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:33.623599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee99f0b-c691-421f-b1f1-88acd4ebca12 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:33.766372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee99f0b-c691-421f-b1f1-88acd4ebca12 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:33.805842 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734c3bc-4e09-493f-bd58-99d8dbfe40b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:33.806433 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734c3bc-4e09-493f-bd58-99d8dbfe40b8 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:33.806714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734c3bc-4e09-493f-bd58-99d8dbfe40b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:33.806990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734c3bc-4e09-493f-bd58-99d8dbfe40b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:33.839387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:33.887896 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734c3bc-4e09-493f-bd58-99d8dbfe40b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:33.889701 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734c3bc-4e09-493f-bd58-99d8dbfe40b8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:34.004160 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 427/871] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:39:33 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:39:34.009583 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cda422-7f6f-41b8-9375-ad1f6459f214 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:34.036427 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:34.133423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cda422-7f6f-41b8-9375-ad1f6459f214 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:34.172128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57a1ed-c153-42d4-b7e8-132a6b99bf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:34.172930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57a1ed-c153-42d4-b7e8-132a6b99bf58 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.173207 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57a1ed-c153-42d4-b7e8-132a6b99bf58 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.173479 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57a1ed-c153-42d4-b7e8-132a6b99bf58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.186301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57a1ed-c153-42d4-b7e8-132a6b99bf58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:34.187771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57a1ed-c153-42d4-b7e8-132a6b99bf58 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:34.210290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-bf57a1ed-c153-42d4-b7e8-132a6b99bf58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:34.210290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:34.210290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:34.210290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:34.210290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:34.210290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:34.210290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:34.210290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:34.210290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:34.210290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:34.210290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:34.210290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:34.210290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:34.210290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:34.210290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:34.211250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:34.212151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:34.212151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:34.212151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:34.212151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:34.215952 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 445/872] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:34.217467 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734c3bc-4e09-493f-bd58-99d8dbfe40b8 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:34.286925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:34.335373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734c3bc-4e09-493f-bd58-99d8dbfe40b8 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:34.375783 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeda98e-08a9-4d74-bf0a-8c7d068b055c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:34.376514 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeda98e-08a9-4d74-bf0a-8c7d068b055c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.376794 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeda98e-08a9-4d74-bf0a-8c7d068b055c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.377066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeda98e-08a9-4d74-bf0a-8c7d068b055c None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.377797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeda98e-08a9-4d74-bf0a-8c7d068b055c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:34.378098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeda98e-08a9-4d74-bf0a-8c7d068b055c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:34.381087 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeda98e-08a9-4d74-bf0a-8c7d068b055c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:34.394887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 428/873] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:34.403166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57a1ed-c153-42d4-b7e8-132a6b99bf58 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:34.468232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9166c-82ca-4332-be57-861300802a0d None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.468914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9166c-82ca-4332-be57-861300802a0d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.469197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9166c-82ca-4332-be57-861300802a0d None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.471366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 429/874] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:39:34.478672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c559a97-2756-4b1b-a6b7-b035ae527639 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.479365 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c559a97-2756-4b1b-a6b7-b035ae527639 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.479788 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c559a97-2756-4b1b-a6b7-b035ae527639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.482769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:34.501188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-4c559a97-2756-4b1b-a6b7-b035ae527639 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:34.502930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-4c559a97-2756-4b1b-a6b7-b035ae527639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:34.522119 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57a1ed-c153-42d4-b7e8-132a6b99bf58 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:34.559565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47d0ca-f519-494f-a10e-b9ff4cc379b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:34.560419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47d0ca-f519-494f-a10e-b9ff4cc379b4 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.560709 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47d0ca-f519-494f-a10e-b9ff4cc379b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.560991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47d0ca-f519-494f-a10e-b9ff4cc379b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.568614 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47d0ca-f519-494f-a10e-b9ff4cc379b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:34.570102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47d0ca-f519-494f-a10e-b9ff4cc379b4 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:34.585332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-3f47d0ca-f519-494f-a10e-b9ff4cc379b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:34.585332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:34.585332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:34.585332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:34.585332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:34.585332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:34.585332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:34.585332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:34.585332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:34.585332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:34.585332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:34.585332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:34.585332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:34.585332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:34.585332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:34.586292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:34.587266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:34.587266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:34.587266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:34.587266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:34.587266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:34.587266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:34.587266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:34.587266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:34.587266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 446/875] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:34.589028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 430/876] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:34.594050 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47d0ca-f519-494f-a10e-b9ff4cc379b4 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:34.613541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c559a97-2756-4b1b-a6b7-b035ae527639 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:34.680742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:34.723180 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47d0ca-f519-494f-a10e-b9ff4cc379b4 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:34.742108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c559a97-2756-4b1b-a6b7-b035ae527639 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:34.751086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:34.761893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cdf66-be26-4252-a7dd-db4c31e334b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:34.762574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cdf66-be26-4252-a7dd-db4c31e334b8 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.762860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cdf66-be26-4252-a7dd-db4c31e334b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.763146 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cdf66-be26-4252-a7dd-db4c31e334b8 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.763992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cdf66-be26-4252-a7dd-db4c31e334b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:34.764306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cdf66-be26-4252-a7dd-db4c31e334b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:34.765750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cdf66-be26-4252-a7dd-db4c31e334b8 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:34.785215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 447/877] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:34.786786 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cdf66-be26-4252-a7dd-db4c31e334b8 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:34.792046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa167dfe-bf50-4de2-9fb4-2482a70824f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:34.793202 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa167dfe-bf50-4de2-9fb4-2482a70824f2 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.793684 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa167dfe-bf50-4de2-9fb4-2482a70824f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.794152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa167dfe-bf50-4de2-9fb4-2482a70824f2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.795015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa167dfe-bf50-4de2-9fb4-2482a70824f2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:34.796600 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa167dfe-bf50-4de2-9fb4-2482a70824f2 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-fa167dfe-bf50-4de2-9fb4-2482a70824f2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:34.810121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:34.811121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:34.811954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:34.811954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:34.811954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:34.811954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:34.811954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:34.811954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:34.811954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:34.811954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:34.811954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:34.811954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:34.811954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:34.811954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:34.811954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:39:34.811954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:34.820469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 431/878] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:39:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:34.821652 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa167dfe-bf50-4de2-9fb4-2482a70824f2 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:34.922189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cdf66-be26-4252-a7dd-db4c31e334b8 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:34.939305 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa167dfe-bf50-4de2-9fb4-2482a70824f2 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:34.955158 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:34.960872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b818d-9426-4ab0-8c3b-fab41d5ea221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:34.961546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b818d-9426-4ab0-8c3b-fab41d5ea221 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.961829 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b818d-9426-4ab0-8c3b-fab41d5ea221 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.962104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b818d-9426-4ab0-8c3b-fab41d5ea221 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.962998 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320b818d-9426-4ab0-8c3b-fab41d5ea221 None admin] RBAC: Adding query filter params (role.id=05f3681d0a644607934e281f75ecd675, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=f0beb65e5dc7469eb17a98dae1f09a0e) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:34.963316 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320b818d-9426-4ab0-8c3b-fab41d5ea221 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:34.965305 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320b818d-9426-4ab0-8c3b-fab41d5ea221 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:34.979752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 448/879] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:39:34 2022] GET /identity/v3/role_assignments?user.id=f0beb65e5dc7469eb17a98dae1f09a0e&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=05f3681d0a644607934e281f75ecd675 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:34.991678 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cf1e6-a2cb-4906-8be8-5993f514e849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:34.992609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cf1e6-a2cb-4906-8be8-5993f514e849 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:34.992899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cf1e6-a2cb-4906-8be8-5993f514e849 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:34.993173 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cf1e6-a2cb-4906-8be8-5993f514e849 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:34.993975 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cf1e6-a2cb-4906-8be8-5993f514e849 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:34.994288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cf1e6-a2cb-4906-8be8-5993f514e849 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:34.995742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cf1e6-a2cb-4906-8be8-5993f514e849 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:35.001888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 432/880] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:39:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:35.007654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b818d-9426-4ab0-8c3b-fab41d5ea221 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:35.066402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:35.131917 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b818d-9426-4ab0-8c3b-fab41d5ea221 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:35.179066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ea30a-bdbd-4a02-9e7d-782591be58bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:35.180124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ea30a-bdbd-4a02-9e7d-782591be58bc None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:35.180551 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ea30a-bdbd-4a02-9e7d-782591be58bc None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:35.180972 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ea30a-bdbd-4a02-9e7d-782591be58bc None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:35.181831 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ea30a-bdbd-4a02-9e7d-782591be58bc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:35.182277 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ea30a-bdbd-4a02-9e7d-782591be58bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:35.183849 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ea30a-bdbd-4a02-9e7d-782591be58bc None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:35.193661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 449/881] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:39:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:35.202916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cf1e6-a2cb-4906-8be8-5993f514e849 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:35.338366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cf1e6-a2cb-4906-8be8-5993f514e849 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:35.378982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ad64a-7b30-4e2f-ae51-8a84ff64daf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:35.379972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ad64a-7b30-4e2f-ae51-8a84ff64daf2 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:35.380263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ad64a-7b30-4e2f-ae51-8a84ff64daf2 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:35.380598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ad64a-7b30-4e2f-ae51-8a84ff64daf2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:35.381828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ad64a-7b30-4e2f-ae51-8a84ff64daf2 None admin] RBAC: Adding query filter params (interface=public, service_id=3d16523950a64fb6a4243a10513eae00, region_id=RegionOne) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:35.382230 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ad64a-7b30-4e2f-ae51-8a84ff64daf2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:35.384019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ad64a-7b30-4e2f-ae51-8a84ff64daf2 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:35.395835 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 433/882] 10.222.0.60 () {56 vars in 1251 bytes} [Thu Jul 21 20:39:35 2022] GET /identity/v3/endpoints?service_id=3d16523950a64fb6a4243a10513eae00&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:35.398577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ea30a-bdbd-4a02-9e7d-782591be58bc None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:35.515821 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ea30a-bdbd-4a02-9e7d-782591be58bc None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:35.531882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:35.555345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1e436-5b03-4ce9-bed1-c6d8b981821a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:35.556057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1e436-5b03-4ce9-bed1-c6d8b981821a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:35.556341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1e436-5b03-4ce9-bed1-c6d8b981821a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:35.556619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1e436-5b03-4ce9-bed1-c6d8b981821a None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:35.557310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1e436-5b03-4ce9-bed1-c6d8b981821a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:35.558831 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1e436-5b03-4ce9-bed1-c6d8b981821a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:35.571456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 450/883] 10.222.0.60 () {56 vars in 1087 bytes} [Thu Jul 21 20:39:35 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:35.698787 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7806f-2962-4a29-9789-c51c8f09a320 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:35.699125 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7806f-2962-4a29-9789-c51c8f09a320 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:35.699345 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7806f-2962-4a29-9789-c51c8f09a320 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:35.701149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 434/884] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:35.707025 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab9b37-1bb0-42e7-933e-1be9007bdada None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:35.707314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab9b37-1bb0-42e7-933e-1be9007bdada None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:35.707665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab9b37-1bb0-42e7-933e-1be9007bdada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:35.726555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-14ab9b37-1bb0-42e7-933e-1be9007bdada None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:35.727830 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-14ab9b37-1bb0-42e7-933e-1be9007bdada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:35.805501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 451/885] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:35.828537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7806f-2962-4a29-9789-c51c8f09a320 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:35.873098 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c2522-7fad-4311-b292-08fb97c1f527 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:35.873374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c2522-7fad-4311-b292-08fb97c1f527 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:35.873664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c2522-7fad-4311-b292-08fb97c1f527 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:35.875184 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 452/886] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:35.883628 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08260ab-3643-47ae-8c85-baa28339621e None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:35.883970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08260ab-3643-47ae-8c85-baa28339621e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:35.884206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08260ab-3643-47ae-8c85-baa28339621e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:35.885178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:35.904548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d08260ab-3643-47ae-8c85-baa28339621e None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:35.905752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d08260ab-3643-47ae-8c85-baa28339621e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:35.962952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7806f-2962-4a29-9789-c51c8f09a320 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:36.007357 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 453/887] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:36.017252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc83162-a92e-4e0d-ba38-424ef3ae953d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:36.018376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc83162-a92e-4e0d-ba38-424ef3ae953d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:36.018849 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc83162-a92e-4e0d-ba38-424ef3ae953d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:36.019341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc83162-a92e-4e0d-ba38-424ef3ae953d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:36.020932 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc83162-a92e-4e0d-ba38-424ef3ae953d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:36.022522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc83162-a92e-4e0d-ba38-424ef3ae953d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:36.026575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08260ab-3643-47ae-8c85-baa28339621e None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-fbc83162-a92e-4e0d-ba38-424ef3ae953d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:36.036914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:36.038174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:36.039155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:36.039155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:36.039155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:36.039155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:36.039155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:36.039155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:36.039155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:36.039155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:36.039155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:36.039155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:36.039155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:36.039155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:36.039155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:39:36.039155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:36.041485 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 435/888] 10.222.0.60 () {56 vars in 1126 bytes} [Thu Jul 21 20:39:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:36.047553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc83162-a92e-4e0d-ba38-424ef3ae953d None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:36.077921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:36.163372 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08260ab-3643-47ae-8c85-baa28339621e None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:36.164619 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc83162-a92e-4e0d-ba38-424ef3ae953d None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:36.172919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:36.207103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fcb341-cdcf-45d6-a25c-7070d07ddad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:36.208485 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fcb341-cdcf-45d6-a25c-7070d07ddad3 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:36.208844 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fcb341-cdcf-45d6-a25c-7070d07ddad3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:36.209153 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fcb341-cdcf-45d6-a25c-7070d07ddad3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:36.209924 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fcb341-cdcf-45d6-a25c-7070d07ddad3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:36.210527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0bcba-33e2-4e5d-bc1e-44aa7025fcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:36.211302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fcb341-cdcf-45d6-a25c-7070d07ddad3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:36.211622 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0bcba-33e2-4e5d-bc1e-44aa7025fcb3 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:36.212125 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0bcba-33e2-4e5d-bc1e-44aa7025fcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:36.212597 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0bcba-33e2-4e5d-bc1e-44aa7025fcb3 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:36.213712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0bcba-33e2-4e5d-bc1e-44aa7025fcb3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:36.214530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0bcba-33e2-4e5d-bc1e-44aa7025fcb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:36.216434 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0bcba-33e2-4e5d-bc1e-44aa7025fcb3 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:36.222211 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 436/889] 10.222.0.60 () {56 vars in 1122 bytes} [Thu Jul 21 20:39:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-c9fcb341-cdcf-45d6-a25c-7070d07ddad3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:36.229636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:36.230613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:36.231437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 454/890] 10.222.0.60 () {56 vars in 1132 bytes} [Thu Jul 21 20:39:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:36.232316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0bcba-33e2-4e5d-bc1e-44aa7025fcb3 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:36.239377 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fcb341-cdcf-45d6-a25c-7070d07ddad3 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:36.357979 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0bcba-33e2-4e5d-bc1e-44aa7025fcb3 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:36.362062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fcb341-cdcf-45d6-a25c-7070d07ddad3 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:36.363694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:36.402269 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50d29c-3b2c-4dc3-a7c0-edbe338f6205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:36.403384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50d29c-3b2c-4dc3-a7c0-edbe338f6205 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:36.404200 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6079cdc-3990-4e56-94bc-2981142498be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:36.404681 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50d29c-3b2c-4dc3-a7c0-edbe338f6205 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:36.404787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6079cdc-3990-4e56-94bc-2981142498be None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:36.404995 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6079cdc-3990-4e56-94bc-2981142498be None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:36.405241 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6079cdc-3990-4e56-94bc-2981142498be None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:36.405673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50d29c-3b2c-4dc3-a7c0-edbe338f6205 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:36.405962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6079cdc-3990-4e56-94bc-2981142498be None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:36.406267 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6079cdc-3990-4e56-94bc-2981142498be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:36.407026 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50d29c-3b2c-4dc3-a7c0-edbe338f6205 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:36.407542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50d29c-3b2c-4dc3-a7c0-edbe338f6205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:36.407642 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6079cdc-3990-4e56-94bc-2981142498be None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:36.409358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50d29c-3b2c-4dc3-a7c0-edbe338f6205 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:36.414715 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 455/891] 10.222.0.60 () {56 vars in 1126 bytes} [Thu Jul 21 20:39:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:36.415522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 437/892] 10.222.0.60 () {56 vars in 1122 bytes} [Thu Jul 21 20:39:36 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:36.427781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6079cdc-3990-4e56-94bc-2981142498be None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:36.560367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6079cdc-3990-4e56-94bc-2981142498be None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:36.605156 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91072b-c97c-485d-8e93-d55b1a141d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:36.605156 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91072b-c97c-485d-8e93-d55b1a141d49 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:36.605897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91072b-c97c-485d-8e93-d55b1a141d49 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:36.605897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91072b-c97c-485d-8e93-d55b1a141d49 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:36.606351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91072b-c97c-485d-8e93-d55b1a141d49 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:36.606664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91072b-c97c-485d-8e93-d55b1a141d49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:36.609304 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91072b-c97c-485d-8e93-d55b1a141d49 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:36.614157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 456/893] 10.222.0.60 () {56 vars in 1126 bytes} [Thu Jul 21 20:39:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:36.619999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50d29c-3b2c-4dc3-a7c0-edbe338f6205 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:36.665980 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:36.744928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50d29c-3b2c-4dc3-a7c0-edbe338f6205 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:36.783891 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696963f-84e0-400d-8518-c3a320a1e54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:36.785001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696963f-84e0-400d-8518-c3a320a1e54e None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:36.786081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696963f-84e0-400d-8518-c3a320a1e54e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:36.786725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696963f-84e0-400d-8518-c3a320a1e54e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:36.787974 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696963f-84e0-400d-8518-c3a320a1e54e None admin] RBAC: Adding query filter params (interface=public, service_id=c21388e454234a559d72f80e468af9d3, region_id=RegionOne) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:36.788525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696963f-84e0-400d-8518-c3a320a1e54e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:36.790195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696963f-84e0-400d-8518-c3a320a1e54e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:36.796992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 438/894] 10.222.0.60 () {56 vars in 1251 bytes} [Thu Jul 21 20:39:36 2022] GET /identity/v3/endpoints?service_id=c21388e454234a559d72f80e468af9d3&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:36.803937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91072b-c97c-485d-8e93-d55b1a141d49 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:36.840422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:36.919059 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91072b-c97c-485d-8e93-d55b1a141d49 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:36.954949 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36858ec5-c456-4d8d-a56a-613addd1d5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:36.956015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36858ec5-c456-4d8d-a56a-613addd1d5ad None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:36.956534 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36858ec5-c456-4d8d-a56a-613addd1d5ad None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:36.957013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36858ec5-c456-4d8d-a56a-613addd1d5ad None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:36.958487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36858ec5-c456-4d8d-a56a-613addd1d5ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:36.960275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36858ec5-c456-4d8d-a56a-613addd1d5ad None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:36.967187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 457/895] 10.222.0.60 () {56 vars in 1087 bytes} [Thu Jul 21 20:39:36 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:37.274315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86268373-7f94-4173-9e2c-394b9e0ae4e0 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:37.274612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86268373-7f94-4173-9e2c-394b9e0ae4e0 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:37.274790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86268373-7f94-4173-9e2c-394b9e0ae4e0 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:37.276496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 439/896] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:37.281989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eba2ed-84dc-40b1-98fa-611c29c83e74 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:37.282275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eba2ed-84dc-40b1-98fa-611c29c83e74 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:37.282562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eba2ed-84dc-40b1-98fa-611c29c83e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:37.300916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-f1eba2ed-84dc-40b1-98fa-611c29c83e74 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:37.302257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f1eba2ed-84dc-40b1-98fa-611c29c83e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:37.393662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 458/897] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:37.420502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86268373-7f94-4173-9e2c-394b9e0ae4e0 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:37.543311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:37.549354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86268373-7f94-4173-9e2c-394b9e0ae4e0 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:37.588910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3330f-d88c-4902-978b-75d5e767b384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:37.590101 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3330f-d88c-4902-978b-75d5e767b384 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:37.590618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3330f-d88c-4902-978b-75d5e767b384 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:37.591167 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3330f-d88c-4902-978b-75d5e767b384 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:37.603773 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3330f-d88c-4902-978b-75d5e767b384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-e7e3330f-d88c-4902-978b-75d5e767b384 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:37.612953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:37.614873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:37.616129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:37.616129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:37.616129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:37.616129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:37.616129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:37.616129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:37.616129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:37.616129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:37.616129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:37.616129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:37.616129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:39:37.616129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:37.618718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 440/898] 10.222.0.60 () {56 vars in 1120 bytes} [Thu Jul 21 20:39:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:37.624638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eba2ed-84dc-40b1-98fa-611c29c83e74 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:37.651490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:37.750216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eba2ed-84dc-40b1-98fa-611c29c83e74 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:37.760997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab4047-4d25-4059-91f3-d90efe01a975 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:37.761307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab4047-4d25-4059-91f3-d90efe01a975 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:37.761482 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab4047-4d25-4059-91f3-d90efe01a975 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:37.763145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 441/899] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:39:37.768525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cea58-3597-4f32-a26e-df1eac555fe2 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:37.768809 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cea58-3597-4f32-a26e-df1eac555fe2 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:37.769088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cea58-3597-4f32-a26e-df1eac555fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:37.789993 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533d7c6-7b56-48a0-880a-120044fd13ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:37.790675 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533d7c6-7b56-48a0-880a-120044fd13ba None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:37.790957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533d7c6-7b56-48a0-880a-120044fd13ba None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:37.791229 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533d7c6-7b56-48a0-880a-120044fd13ba None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:37.791958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-6d8cea58-3597-4f32-a26e-df1eac555fe2 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:37.792069 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533d7c6-7b56-48a0-880a-120044fd13ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:37.792319 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533d7c6-7b56-48a0-880a-120044fd13ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:37.793769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533d7c6-7b56-48a0-880a-120044fd13ba None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:37.793888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6d8cea58-3597-4f32-a26e-df1eac555fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:37.803032 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 459/900] 10.222.0.60 () {56 vars in 1115 bytes} [Thu Jul 21 20:39:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:37.809011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533d7c6-7b56-48a0-880a-120044fd13ba None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:37.879982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 442/901] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:37.896735 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cea58-3597-4f32-a26e-df1eac555fe2 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:37.912551 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:37.932635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533d7c6-7b56-48a0-880a-120044fd13ba None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:37.971158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631a09e-a03b-4483-bf9e-0290e4b45177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:37.972004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631a09e-a03b-4483-bf9e-0290e4b45177 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:37.972287 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631a09e-a03b-4483-bf9e-0290e4b45177 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:37.972560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631a09e-a03b-4483-bf9e-0290e4b45177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:37.985323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631a09e-a03b-4483-bf9e-0290e4b45177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:37.986782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631a09e-a03b-4483-bf9e-0290e4b45177 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:37.994577 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:38.002478 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cea58-3597-4f32-a26e-df1eac555fe2 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:38.005526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-7631a09e-a03b-4483-bf9e-0290e4b45177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:38.005526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:38.005526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:38.005526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:38.005526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:38.005526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:38.005526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:38.005526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:38.005526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:38.005526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:38.005526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:38.005526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:38.005526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:38.005526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:38.005526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:38.006402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:38.007272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:38.007272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:38.007272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:38.007272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:38.007272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 460/902] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:38.016723 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631a09e-a03b-4483-bf9e-0290e4b45177 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:38.040193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f37d5c-bbfe-4e53-a0a1-86527622a6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:38.040999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f37d5c-bbfe-4e53-a0a1-86527622a6dd None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:38.041279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f37d5c-bbfe-4e53-a0a1-86527622a6dd None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:38.041552 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f37d5c-bbfe-4e53-a0a1-86527622a6dd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:38.042271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f37d5c-bbfe-4e53-a0a1-86527622a6dd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:38.043653 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f37d5c-bbfe-4e53-a0a1-86527622a6dd None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-96f37d5c-bbfe-4e53-a0a1-86527622a6dd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:38.056053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:38.057071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:38.057906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:38.057906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:38.057906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:38.057906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:38.057906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:38.057906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:38.057906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:38.057906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:38.057906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:38.057906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:38.057906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:38.057906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:38.057906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 20:39:38.057906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:38.062239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 443/903] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:39:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:38.063180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f37d5c-bbfe-4e53-a0a1-86527622a6dd None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:38.148544 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631a09e-a03b-4483-bf9e-0290e4b45177 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:38.187592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f37d5c-bbfe-4e53-a0a1-86527622a6dd None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:38.194277 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9971c-2379-4d98-92a4-38338752a1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:38.194277 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9971c-2379-4d98-92a4-38338752a1a9 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:38.194840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9971c-2379-4d98-92a4-38338752a1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:38.195360 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9971c-2379-4d98-92a4-38338752a1a9 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:38.199969 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9971c-2379-4d98-92a4-38338752a1a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:38.200571 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9971c-2379-4d98-92a4-38338752a1a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:38.202923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9971c-2379-4d98-92a4-38338752a1a9 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:38.214717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:38.229350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 461/904] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:38.230909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9971c-2379-4d98-92a4-38338752a1a9 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:38.239032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db5f18-70c8-48f4-8d11-a5e2dd00ec13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:38.239842 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db5f18-70c8-48f4-8d11-a5e2dd00ec13 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:38.240141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db5f18-70c8-48f4-8d11-a5e2dd00ec13 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:38.240434 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db5f18-70c8-48f4-8d11-a5e2dd00ec13 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:38.241145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db5f18-70c8-48f4-8d11-a5e2dd00ec13 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:38.241451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db5f18-70c8-48f4-8d11-a5e2dd00ec13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:38.242970 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db5f18-70c8-48f4-8d11-a5e2dd00ec13 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:38.255901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 444/905] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:39:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:38.257419 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db5f18-70c8-48f4-8d11-a5e2dd00ec13 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:38.280359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:38.370422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9971c-2379-4d98-92a4-38338752a1a9 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:38.396035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db5f18-70c8-48f4-8d11-a5e2dd00ec13 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:38.424336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1c4fa-9edd-49a1-ac80-10230aedceda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:38.425696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1c4fa-9edd-49a1-ac80-10230aedceda None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:38.426747 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1c4fa-9edd-49a1-ac80-10230aedceda None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:38.427232 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1c4fa-9edd-49a1-ac80-10230aedceda None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:38.436195 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1c4fa-9edd-49a1-ac80-10230aedceda None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:38.438495 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1c4fa-9edd-49a1-ac80-10230aedceda None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:38.440240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bd3ba-ff6a-4ab2-b921-b1ce2a8910cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:38.440999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bd3ba-ff6a-4ab2-b921-b1ce2a8910cd None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:38.441647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bd3ba-ff6a-4ab2-b921-b1ce2a8910cd None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:38.442098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bd3ba-ff6a-4ab2-b921-b1ce2a8910cd None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:38.443847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bd3ba-ff6a-4ab2-b921-b1ce2a8910cd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:38.444240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bd3ba-ff6a-4ab2-b921-b1ce2a8910cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:38.445706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bd3ba-ff6a-4ab2-b921-b1ce2a8910cd None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:38.455762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 445/906] 10.222.0.60 () {56 vars in 1114 bytes} [Thu Jul 21 20:39:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:38.457307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bd3ba-ff6a-4ab2-b921-b1ce2a8910cd None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-b8e1c4fa-9edd-49a1-ac80-10230aedceda None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:38.463020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:38.464024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:38.464943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:38.464943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:38.464943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:38.464943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:38.464943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:38.464943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:38.464943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:38.464943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:38.464943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:39:38.464943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:38.468860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 462/907] 10.222.0.60 () {56 vars in 1117 bytes} [Thu Jul 21 20:39:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:38.469947 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1c4fa-9edd-49a1-ac80-10230aedceda None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:38.544813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:38.567612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:38.590283 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bd3ba-ff6a-4ab2-b921-b1ce2a8910cd None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:38.595053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1c4fa-9edd-49a1-ac80-10230aedceda None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:38.629964 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26040434-120a-430e-b589-2221e9110496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:38.631145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26040434-120a-430e-b589-2221e9110496 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:38.631859 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26040434-120a-430e-b589-2221e9110496 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:38.632214 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26040434-120a-430e-b589-2221e9110496 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:38.632972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26040434-120a-430e-b589-2221e9110496 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:38.634614 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26040434-120a-430e-b589-2221e9110496 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:38.635773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2dee1-16f3-48d8-b1b2-40eacb9ac510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:38.636964 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2dee1-16f3-48d8-b1b2-40eacb9ac510 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:38.637402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2dee1-16f3-48d8-b1b2-40eacb9ac510 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:38.637852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2dee1-16f3-48d8-b1b2-40eacb9ac510 None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:38.638857 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2dee1-16f3-48d8-b1b2-40eacb9ac510 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:38.639318 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2dee1-16f3-48d8-b1b2-40eacb9ac510 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:38.641024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2dee1-16f3-48d8-b1b2-40eacb9ac510 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:38.663949 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 463/909] 10.222.0.60 () {56 vars in 1149 bytes} [Thu Jul 21 20:39:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:38.664171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 446/909] 10.222.0.60 () {60 vars in 1144 bytes} [Thu Jul 21 20:39:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:39:38.675883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26040434-120a-430e-b589-2221e9110496 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:38.784656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f78cb-3dcc-4513-a02f-ef189575dc65 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:38.785014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f78cb-3dcc-4513-a02f-ef189575dc65 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:38.785543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f78cb-3dcc-4513-a02f-ef189575dc65 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:38.788190 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 464/910] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:39:38.796378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb03c2b-99c4-4d74-b3a5-dd5438f29d81 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:38.796831 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb03c2b-99c4-4d74-b3a5-dd5438f29d81 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:38.797209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb03c2b-99c4-4d74-b3a5-dd5438f29d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:38.819775 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26040434-120a-430e-b589-2221e9110496 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:38.821924 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-6eb03c2b-99c4-4d74-b3a5-dd5438f29d81 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:38.823142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6eb03c2b-99c4-4d74-b3a5-dd5438f29d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:38.866853 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:38.873441 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb77e1-8e7e-46b8-a1d2-f1171788d8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:38.874245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb77e1-8e7e-46b8-a1d2-f1171788d8c9 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:38.874356 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb77e1-8e7e-46b8-a1d2-f1171788d8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:38.874633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb77e1-8e7e-46b8-a1d2-f1171788d8c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:38.927692 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 465/911] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:38.965060 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb03c2b-99c4-4d74-b3a5-dd5438f29d81 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:38.991910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb77e1-8e7e-46b8-a1d2-f1171788d8c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:38.993939 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb77e1-8e7e-46b8-a1d2-f1171788d8c9 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:39.029141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:39.109196 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb03c2b-99c4-4d74-b3a5-dd5438f29d81 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:39.136104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 447/912] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:39:38 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:39:39.137538 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb77e1-8e7e-46b8-a1d2-f1171788d8c9 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:39.155060 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571234a7-9c3d-4ccb-8a4d-f0d4ca01f2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:39.155773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571234a7-9c3d-4ccb-8a4d-f0d4ca01f2e3 None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:39.156088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571234a7-9c3d-4ccb-8a4d-f0d4ca01f2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:39.156399 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571234a7-9c3d-4ccb-8a4d-f0d4ca01f2e3 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:39.157158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571234a7-9c3d-4ccb-8a4d-f0d4ca01f2e3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:39.158555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571234a7-9c3d-4ccb-8a4d-f0d4ca01f2e3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:39.184060 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 466/913] 10.222.0.60 () {60 vars in 1141 bytes} [Thu Jul 21 20:39:38 2022] POST /identity/v3/services => generated 241 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:39.187596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b21587-7e44-40e4-880c-dc8a2ad338ff None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:39.187961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b21587-7e44-40e4-880c-dc8a2ad338ff None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:39.188257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b21587-7e44-40e4-880c-dc8a2ad338ff None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:39.191216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:39.194814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 467/914] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:39:39.196331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011cf08-e36d-4247-afb8-32b5f9abdf60 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:39.196686 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011cf08-e36d-4247-afb8-32b5f9abdf60 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:39.196925 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011cf08-e36d-4247-afb8-32b5f9abdf60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:39.217313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-9011cf08-e36d-4247-afb8-32b5f9abdf60 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:39.218543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9011cf08-e36d-4247-afb8-32b5f9abdf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:39.277203 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb77e1-8e7e-46b8-a1d2-f1171788d8c9 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:39.330533 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44872a2-a074-453f-8205-fb106a5b2491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:39.330756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 468/915] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:39.339988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44872a2-a074-453f-8205-fb106a5b2491 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:39.339988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44872a2-a074-453f-8205-fb106a5b2491 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:39.339988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44872a2-a074-453f-8205-fb106a5b2491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:39.343740 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44872a2-a074-453f-8205-fb106a5b2491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:39.345302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44872a2-a074-453f-8205-fb106a5b2491 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:39.357654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011cf08-e36d-4247-afb8-32b5f9abdf60 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:39.366590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-e44872a2-a074-453f-8205-fb106a5b2491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:39.366590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:39.366590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:39.366590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:39.366590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:39.366590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:39.366590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:39.366590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:39.366590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:39.366590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:39.366590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:39.366590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:39.366590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:39.366590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:39.366590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:39.367557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:39.368961 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:39.368961 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:39.368961 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:39.368961 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:39.368961 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 448/916] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:39.373924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44872a2-a074-453f-8205-fb106a5b2491 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:39.402866 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:39.500218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:39.512178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44872a2-a074-453f-8205-fb106a5b2491 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:39.513623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011cf08-e36d-4247-afb8-32b5f9abdf60 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:39.555320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347fe0b-30b8-475b-90a6-157e99663aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:39.556279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347fe0b-30b8-475b-90a6-157e99663aef None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:39.556637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347fe0b-30b8-475b-90a6-157e99663aef None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:39.556990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347fe0b-30b8-475b-90a6-157e99663aef None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:39.557637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b0564-4fb1-4352-8040-48366cd976fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:39.557763 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347fe0b-30b8-475b-90a6-157e99663aef None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:39.558303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b0564-4fb1-4352-8040-48366cd976fa None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:39.558594 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b0564-4fb1-4352-8040-48366cd976fa None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:39.558864 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b0564-4fb1-4352-8040-48366cd976fa None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:39.559480 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347fe0b-30b8-475b-90a6-157e99663aef None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:39.559588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b0564-4fb1-4352-8040-48366cd976fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:39.559924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b0564-4fb1-4352-8040-48366cd976fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:39.561669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b0564-4fb1-4352-8040-48366cd976fa None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-d347fe0b-30b8-475b-90a6-157e99663aef None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:39.575244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:39.576449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:39.577458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:39.577458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:39.577458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:39.577458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:39.577458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:39.577458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:39.577458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:39.577458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:39.577458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:39.577458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:39.577458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:39.577458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:39.577458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:39.577458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 20:39:39.577458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:39.578417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 449/917] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:39.578511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 469/918] 10.222.0.60 () {56 vars in 1132 bytes} [Thu Jul 21 20:39:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:39.584197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b0564-4fb1-4352-8040-48366cd976fa None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:39.586586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347fe0b-30b8-475b-90a6-157e99663aef None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:39.718661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:39.729366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347fe0b-30b8-475b-90a6-157e99663aef None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:39.733456 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b0564-4fb1-4352-8040-48366cd976fa None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:39.769888 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021448c-8edb-48a1-a81b-ba7dd0456439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:39.771218 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021448c-8edb-48a1-a81b-ba7dd0456439 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:39.771707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021448c-8edb-48a1-a81b-ba7dd0456439 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:39.772222 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021448c-8edb-48a1-a81b-ba7dd0456439 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:39.773168 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021448c-8edb-48a1-a81b-ba7dd0456439 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:39.774909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021448c-8edb-48a1-a81b-ba7dd0456439 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:39.775588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021448c-8edb-48a1-a81b-ba7dd0456439 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:39.782215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 470/919] 10.222.0.60 () {56 vars in 1126 bytes} [Thu Jul 21 20:39:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:39.783327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1388885-7855-4695-9c00-eefab2bf06be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:39.787860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1388885-7855-4695-9c00-eefab2bf06be None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:39.787860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1388885-7855-4695-9c00-eefab2bf06be None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:39.787860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1388885-7855-4695-9c00-eefab2bf06be None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:39.795553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021448c-8edb-48a1-a81b-ba7dd0456439 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:39.796395 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1388885-7855-4695-9c00-eefab2bf06be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:39.797941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1388885-7855-4695-9c00-eefab2bf06be None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:39.813813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-d1388885-7855-4695-9c00-eefab2bf06be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:39.813813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:39.813813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:39.813813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:39.813813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:39.813813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:39.813813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:39.813813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:39.813813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:39.813813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:39.813813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:39.813813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:39.813813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:39.813813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:39.813813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:39.814752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:39.815647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:39.815647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:39.815647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:39.815647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:39.815647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:39.815647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:39.815647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:39.815647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:39.815647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:39.816324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 450/920] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:39.821488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1388885-7855-4695-9c00-eefab2bf06be None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:39.946115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1388885-7855-4695-9c00-eefab2bf06be None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:39.952111 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021448c-8edb-48a1-a81b-ba7dd0456439 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:39.990405 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b37a8-d879-4ec8-9701-f536f4a3309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:39.991081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b37a8-d879-4ec8-9701-f536f4a3309a None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:39.991420 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b37a8-d879-4ec8-9701-f536f4a3309a None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:39.991700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b37a8-d879-4ec8-9701-f536f4a3309a None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:39.992643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b37a8-d879-4ec8-9701-f536f4a3309a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:39.992951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b37a8-d879-4ec8-9701-f536f4a3309a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:39.994487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b37a8-d879-4ec8-9701-f536f4a3309a None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:39.999428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7ff5c-6def-4098-b119-89fc06e9d191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:40.000369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7ff5c-6def-4098-b119-89fc06e9d191 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:40.000868 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7ff5c-6def-4098-b119-89fc06e9d191 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:40.001425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7ff5c-6def-4098-b119-89fc06e9d191 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:40.002938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7ff5c-6def-4098-b119-89fc06e9d191 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:40.003472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7ff5c-6def-4098-b119-89fc06e9d191 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:40.006179 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7ff5c-6def-4098-b119-89fc06e9d191 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:40.012456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 471/922] 10.222.0.60 () {56 vars in 1126 bytes} [Thu Jul 21 20:39:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:40.015074 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7ff5c-6def-4098-b119-89fc06e9d191 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:40.015971 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 451/922] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:40.021702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b37a8-d879-4ec8-9701-f536f4a3309a None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:40.047545 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:40.108200 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:40.149361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7ff5c-6def-4098-b119-89fc06e9d191 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:40.183454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b37a8-d879-4ec8-9701-f536f4a3309a None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:40.189587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555a4b9-5ed6-4b06-b8a6-37918e9e97b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:40.190386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555a4b9-5ed6-4b06-b8a6-37918e9e97b9 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:40.190814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555a4b9-5ed6-4b06-b8a6-37918e9e97b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:40.191226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555a4b9-5ed6-4b06-b8a6-37918e9e97b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:40.193129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555a4b9-5ed6-4b06-b8a6-37918e9e97b9 None admin] RBAC: Adding query filter params (role.id=952e60f3db5044afa8b3043155a1ccca, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=f0beb65e5dc7469eb17a98dae1f09a0e) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:40.193826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555a4b9-5ed6-4b06-b8a6-37918e9e97b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:40.196650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555a4b9-5ed6-4b06-b8a6-37918e9e97b9 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:40.212219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 472/923] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:39:40 2022] GET /identity/v3/role_assignments?user.id=f0beb65e5dc7469eb17a98dae1f09a0e&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=952e60f3db5044afa8b3043155a1ccca => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:40.233416 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e97ff-78bb-44ec-85af-75c7c7357953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:40.234695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e97ff-78bb-44ec-85af-75c7c7357953 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:40.235452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e97ff-78bb-44ec-85af-75c7c7357953 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:40.236265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e97ff-78bb-44ec-85af-75c7c7357953 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:40.237945 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e97ff-78bb-44ec-85af-75c7c7357953 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:40.239919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e97ff-78bb-44ec-85af-75c7c7357953 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:40.263582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 452/924] 10.222.0.60 () {60 vars in 1144 bytes} [Thu Jul 21 20:39:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:40.883256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903075-5e14-4be2-aa10-c799297cd95f None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:40.883804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903075-5e14-4be2-aa10-c799297cd95f None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:40.883927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903075-5e14-4be2-aa10-c799297cd95f None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:40.885482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 473/925] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:40.891201 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc5dd0-7dc1-4a2b-bd2b-feebaffcfb78 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:40.891675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc5dd0-7dc1-4a2b-bd2b-feebaffcfb78 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:40.891781 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc5dd0-7dc1-4a2b-bd2b-feebaffcfb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:40.909803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c4cc5dd0-7dc1-4a2b-bd2b-feebaffcfb78 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:40.911074 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c4cc5dd0-7dc1-4a2b-bd2b-feebaffcfb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:40.994123 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 453/926] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:41.017590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91903075-5e14-4be2-aa10-c799297cd95f None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:41.127983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:41.146026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91903075-5e14-4be2-aa10-c799297cd95f None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:41.183569 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ebab0-7a70-4d9f-8748-bdf9fa20d02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:41.184265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ebab0-7a70-4d9f-8748-bdf9fa20d02f None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:41.184548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ebab0-7a70-4d9f-8748-bdf9fa20d02f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:41.184821 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ebab0-7a70-4d9f-8748-bdf9fa20d02f None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:41.185541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ebab0-7a70-4d9f-8748-bdf9fa20d02f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:41.186972 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ebab0-7a70-4d9f-8748-bdf9fa20d02f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:41.188375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: WARNING keystone.server.flask.common [None req-441ebab0-7a70-4d9f-8748-bdf9fa20d02f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 20:39:41.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.sql.core [None req-441ebab0-7a70-4d9f-8748-bdf9fa20d02f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 20:39:41.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 20:39:41.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [parameters: {'id': 'e974758f6b70454fb5f224e5da0fce29', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 20:39:41.208600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87061) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 20:39:41.219854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-441ebab0-7a70-4d9f-8748-bdf9fa20d02f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e974758f6b70454fb5f224e5da0fce29.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e974758f6b70454fb5f224e5da0fce29. Jul 21 20:39:41.219854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:41.219854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 20:39:41.219854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 20:39:41.219854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 20:39:41.219854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 20:39:41.219854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 20:39:41.219854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 20:39:41.219854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 20:39:41.219854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application conn.query(q) Jul 21 20:39:41.219854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 20:39:41.219854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 20:39:41.219854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application result.read() Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 21 20:39:41.220706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application next(self.gen) Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application yield resource Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application next(self.gen) Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise self.value Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application session.commit() Jul 21 20:39:41.221618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self.session.flush() Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application compat.raise_( Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 20:39:41.222474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 20:39:41.223923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 20:39:41.223923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 20:39:41.223923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 20:39:41.223923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 20:39:41.223923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 20:39:41.223923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 20:39:41.223923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 20:39:41.223923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 20:39:41.223923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 20:39:41.223923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 20:39:41.223923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 20:39:41.223923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 20:39:41.223923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 20:39:41.223923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 20:39:41.223923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application conn.query(q) Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application result.read() Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 20:39:41.224687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 20:39:41.225542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 20:39:41.225542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 20:39:41.225542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 20:39:41.225542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 20:39:41.225542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 20:39:41.225542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 20:39:41.225542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 20:39:41.225542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 20:39:41.225542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [parameters: {'id': 'e974758f6b70454fb5f224e5da0fce29', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 20:39:41.225542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 20:39:41.225542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:41.225542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:41.225542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:41.225542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:41.225542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:41.226328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:41.227224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:41.227224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 21 20:39:41.227224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 21 20:39:41.227224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:41.227224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:41.227224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 21 20:39:41.227224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 21 20:39:41.227224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e974758f6b70454fb5f224e5da0fce29. Jul 21 20:39:41.227224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:41.227224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 474/927] 10.222.0.60 () {60 vars in 1140 bytes} [Thu Jul 21 20:39:41 2022] POST /identity/v3/projects => generated 262 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:41.232790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc5dd0-7dc1-4a2b-bd2b-feebaffcfb78 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:41.276142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:41.338096 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc5dd0-7dc1-4a2b-bd2b-feebaffcfb78 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:41.376112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd945a-a6fd-4da5-b458-07a362699357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:41.377329 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd945a-a6fd-4da5-b458-07a362699357 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:41.377820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd945a-a6fd-4da5-b458-07a362699357 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:41.378302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd945a-a6fd-4da5-b458-07a362699357 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:41.386612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd945a-a6fd-4da5-b458-07a362699357 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:41.388542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd945a-a6fd-4da5-b458-07a362699357 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:41.404807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-01cd945a-a6fd-4da5-b458-07a362699357 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:41.404807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:41.404807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:41.404807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:41.404807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:41.404807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:41.404807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:41.404807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:41.404807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:41.404807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:41.404807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:41.404807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:41.404807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:41.404807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:41.404807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:41.405718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:41.407114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:41.407114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:41.407114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:41.407114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:41.407114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:41.407114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:41.407114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:41.407114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:41.409738 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 454/928] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:41.416298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ebab0-7a70-4d9f-8748-bdf9fa20d02f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:41.498530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c722cb-2953-45d6-a4f5-991f8ae14d4f None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:41.499051 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c722cb-2953-45d6-a4f5-991f8ae14d4f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:41.499539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c722cb-2953-45d6-a4f5-991f8ae14d4f None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:41.501426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 455/929] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:41.507592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edecc1c-e68f-4748-9831-b3fa3b5f62e4 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:41.508181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edecc1c-e68f-4748-9831-b3fa3b5f62e4 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:41.508672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edecc1c-e68f-4748-9831-b3fa3b5f62e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:41.529644 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-5edecc1c-e68f-4748-9831-b3fa3b5f62e4 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:41.531243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5edecc1c-e68f-4748-9831-b3fa3b5f62e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:41.554117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ebab0-7a70-4d9f-8748-bdf9fa20d02f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:41.612086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f10de2-ec92-45df-86cc-6a565c639070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:41.612956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f10de2-ec92-45df-86cc-6a565c639070 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:41.613341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f10de2-ec92-45df-86cc-6a565c639070 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:41.613936 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f10de2-ec92-45df-86cc-6a565c639070 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:41.615068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f10de2-ec92-45df-86cc-6a565c639070 None admin] RBAC: Adding query filter params (name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:41.615482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f10de2-ec92-45df-86cc-6a565c639070 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:41.617282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f10de2-ec92-45df-86cc-6a565c639070 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:41.624000 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 456/930] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:41.638785 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 475/931] 10.222.0.60 () {56 vars in 1112 bytes} [Thu Jul 21 20:39:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:41.643995 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edecc1c-e68f-4748-9831-b3fa3b5f62e4 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:41.682933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:41.765638 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edecc1c-e68f-4748-9831-b3fa3b5f62e4 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:41.805507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709f004-fd2c-4144-a215-46d5c3caa7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:41.806320 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709f004-fd2c-4144-a215-46d5c3caa7a0 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:41.806602 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709f004-fd2c-4144-a215-46d5c3caa7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:41.806876 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709f004-fd2c-4144-a215-46d5c3caa7a0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:41.807526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709f004-fd2c-4144-a215-46d5c3caa7a0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:41.809011 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709f004-fd2c-4144-a215-46d5c3caa7a0 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-5709f004-fd2c-4144-a215-46d5c3caa7a0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:41.822411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:41.823454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:41.824384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:41.824384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:41.824384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:41.824384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:41.824384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:41.824384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:41.824384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:41.824384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:41.824384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:41.824384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:41.824384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:41.824384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:41.824384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:39:41.824384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:41.824384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 457/932] 10.222.0.60 () {56 vars in 1126 bytes} [Thu Jul 21 20:39:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:41.830138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f10de2-ec92-45df-86cc-6a565c639070 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:41.834778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:41.946972 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f10de2-ec92-45df-86cc-6a565c639070 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:41.988230 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b5bb8-2b44-477c-8d1d-c960c0f89207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:41.988976 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b5bb8-2b44-477c-8d1d-c960c0f89207 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:41.989302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b5bb8-2b44-477c-8d1d-c960c0f89207 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:41.989581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b5bb8-2b44-477c-8d1d-c960c0f89207 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:41.990300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b5bb8-2b44-477c-8d1d-c960c0f89207 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:41.990600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b5bb8-2b44-477c-8d1d-c960c0f89207 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:41.992011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b5bb8-2b44-477c-8d1d-c960c0f89207 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:41.998031 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 476/933] 10.222.0.60 () {56 vars in 1122 bytes} [Thu Jul 21 20:39:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:42.003942 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709f004-fd2c-4144-a215-46d5c3caa7a0 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:42.117938 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:42.126376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709f004-fd2c-4144-a215-46d5c3caa7a0 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:42.166260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206eb08-6f7c-449b-8835-cef5890f9d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:42.166938 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206eb08-6f7c-449b-8835-cef5890f9d3d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:42.167219 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206eb08-6f7c-449b-8835-cef5890f9d3d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:42.167496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206eb08-6f7c-449b-8835-cef5890f9d3d None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:42.168306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206eb08-6f7c-449b-8835-cef5890f9d3d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:42.168614 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206eb08-6f7c-449b-8835-cef5890f9d3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:42.170104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206eb08-6f7c-449b-8835-cef5890f9d3d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:42.175891 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 458/934] 10.222.0.60 () {56 vars in 1122 bytes} [Thu Jul 21 20:39:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:42.182187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b5bb8-2b44-477c-8d1d-c960c0f89207 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:42.254817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:42.295565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b5bb8-2b44-477c-8d1d-c960c0f89207 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:42.332063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7055933-1018-41f4-a5b9-369775be9e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:42.332775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7055933-1018-41f4-a5b9-369775be9e9d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:42.333053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7055933-1018-41f4-a5b9-369775be9e9d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:42.333319 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7055933-1018-41f4-a5b9-369775be9e9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:42.334113 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7055933-1018-41f4-a5b9-369775be9e9d None admin] RBAC: Adding query filter params (interface=public, service_id=b30ddcac40b04bc083b45252ed10a423, region_id=RegionOne) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:42.334425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7055933-1018-41f4-a5b9-369775be9e9d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:42.335824 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7055933-1018-41f4-a5b9-369775be9e9d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:42.341930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 477/935] 10.222.0.60 () {56 vars in 1251 bytes} [Thu Jul 21 20:39:42 2022] GET /identity/v3/endpoints?service_id=b30ddcac40b04bc083b45252ed10a423&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:42.347504 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206eb08-6f7c-449b-8835-cef5890f9d3d None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:42.454952 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3afc9-46f7-4b39-955b-5c858e82340f None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:42.455409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3afc9-46f7-4b39-955b-5c858e82340f None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:42.455773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3afc9-46f7-4b39-955b-5c858e82340f None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:42.457488 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 478/936] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:42.462774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66965b66-c41c-4356-a721-c36b6119519d None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:42.463381 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66965b66-c41c-4356-a721-c36b6119519d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:42.463900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66965b66-c41c-4356-a721-c36b6119519d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:42.464655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206eb08-6f7c-449b-8835-cef5890f9d3d None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:42.486844 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-66965b66-c41c-4356-a721-c36b6119519d None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:42.489326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-66965b66-c41c-4356-a721-c36b6119519d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:42.502205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcfdd9-da58-4617-ab2e-f045039a92d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:42.503135 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcfdd9-da58-4617-ab2e-f045039a92d0 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:42.503571 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcfdd9-da58-4617-ab2e-f045039a92d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:42.504028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcfdd9-da58-4617-ab2e-f045039a92d0 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:42.505767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcfdd9-da58-4617-ab2e-f045039a92d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:42.507333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcfdd9-da58-4617-ab2e-f045039a92d0 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:42.514367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 459/937] 10.222.0.60 () {56 vars in 1087 bytes} [Thu Jul 21 20:39:42 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:42.529279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a09ff7-a29b-4ca5-abaf-0e5b0b75b05f None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:42.529806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a09ff7-a29b-4ca5-abaf-0e5b0b75b05f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:42.530260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a09ff7-a29b-4ca5-abaf-0e5b0b75b05f None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:42.532029 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 460/938] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:39:42.539407 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bf226-b429-4fab-8d34-6f0b668fe0b9 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:42.540274 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bf226-b429-4fab-8d34-6f0b668fe0b9 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:42.540733 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105bf226-b429-4fab-8d34-6f0b668fe0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:42.567448 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-105bf226-b429-4fab-8d34-6f0b668fe0b9 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:42.569099 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-105bf226-b429-4fab-8d34-6f0b668fe0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:42.589097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 479/939] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:42.591494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:42.617437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66965b66-c41c-4356-a721-c36b6119519d None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:42.676572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 461/940] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:42.704021 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bf226-b429-4fab-8d34-6f0b668fe0b9 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:42.710216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:42.766209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66965b66-c41c-4356-a721-c36b6119519d None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:42.816228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7efc73-0d18-4be3-b492-a6c0d9eb9584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:42.817110 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7efc73-0d18-4be3-b492-a6c0d9eb9584 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:42.817419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7efc73-0d18-4be3-b492-a6c0d9eb9584 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:42.817692 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7efc73-0d18-4be3-b492-a6c0d9eb9584 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:42.821935 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105bf226-b429-4fab-8d34-6f0b668fe0b9 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:42.830270 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7efc73-0d18-4be3-b492-a6c0d9eb9584 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-7f7efc73-0d18-4be3-b492-a6c0d9eb9584 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:42.839265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:42.840490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:42.841531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:42.841531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:42.841531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:42.841531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:42.841531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:42.841531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:42.841531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:42.841531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:42.841531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:42.841531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:42.841531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:39:42.841531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:42.843892 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 480/941] 10.222.0.60 () {56 vars in 1120 bytes} [Thu Jul 21 20:39:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:42.855745 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7efc73-0d18-4be3-b492-a6c0d9eb9584 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:42.863159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb995b-954f-4129-bd86-68701d561116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:42.864524 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb995b-954f-4129-bd86-68701d561116 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:42.865239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb995b-954f-4129-bd86-68701d561116 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:42.865683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb995b-954f-4129-bd86-68701d561116 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:42.866513 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb995b-954f-4129-bd86-68701d561116 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:42.868311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb995b-954f-4129-bd86-68701d561116 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-12bb995b-954f-4129-bd86-68701d561116 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:42.882132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:42.883414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:42.884371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:42.884371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:42.884371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:42.884371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:42.884371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:42.884371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:42.884371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:42.884371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:42.884371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:42.884371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:42.884371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:42.884371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:42.884371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:39:42.884371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:42.891565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 462/942] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:42.893152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb995b-954f-4129-bd86-68701d561116 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:42.917010 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:42.981048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7efc73-0d18-4be3-b492-a6c0d9eb9584 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:43.005173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:43.017873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb995b-954f-4129-bd86-68701d561116 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:43.018403 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4d2c2-22af-4867-8a7e-f18686e1ab6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:43.018598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4d2c2-22af-4867-8a7e-f18686e1ab6b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:43.018887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4d2c2-22af-4867-8a7e-f18686e1ab6b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:43.019168 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4d2c2-22af-4867-8a7e-f18686e1ab6b None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:43.020541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4d2c2-22af-4867-8a7e-f18686e1ab6b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:43.020937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4d2c2-22af-4867-8a7e-f18686e1ab6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:43.023328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4d2c2-22af-4867-8a7e-f18686e1ab6b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:43.036936 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 481/943] 10.222.0.60 () {56 vars in 1115 bytes} [Thu Jul 21 20:39:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:43.038054 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4d2c2-22af-4867-8a7e-f18686e1ab6b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:43.059661 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f73c4-137a-4819-a544-75123142e32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:43.060649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f73c4-137a-4819-a544-75123142e32b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:43.061209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f73c4-137a-4819-a544-75123142e32b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:43.061661 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f73c4-137a-4819-a544-75123142e32b None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:43.062519 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f73c4-137a-4819-a544-75123142e32b None admin] RBAC: Adding query filter params (name=compute) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:43.062969 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f73c4-137a-4819-a544-75123142e32b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:43.064817 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f73c4-137a-4819-a544-75123142e32b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:43.075667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 463/944] 10.222.0.60 () {56 vars in 1112 bytes} [Thu Jul 21 20:39:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:43.077413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f73c4-137a-4819-a544-75123142e32b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:43.155881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4d2c2-22af-4867-8a7e-f18686e1ab6b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:43.172281 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:43.195106 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f73c4-137a-4819-a544-75123142e32b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:43.195693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2275e9-3ae8-4085-9a37-dd1d56ec3d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:43.196604 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2275e9-3ae8-4085-9a37-dd1d56ec3d98 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:43.196891 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2275e9-3ae8-4085-9a37-dd1d56ec3d98 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:43.197224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2275e9-3ae8-4085-9a37-dd1d56ec3d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:43.210661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2275e9-3ae8-4085-9a37-dd1d56ec3d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:43.212519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2275e9-3ae8-4085-9a37-dd1d56ec3d98 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:43.232087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-8e2275e9-3ae8-4085-9a37-dd1d56ec3d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:43.232087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:43.232087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:43.232087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:43.232087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:43.232087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:43.232087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:43.232087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:43.232087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:43.232087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:43.232087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:43.232087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:43.232087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:43.232087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:43.232087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:43.233227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:43.234288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:43.234288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:43.234288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:43.234288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:43.236636 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938bbaf-67d6-484c-91d6-962dce132dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:43.237464 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938bbaf-67d6-484c-91d6-962dce132dfd None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:43.237899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938bbaf-67d6-484c-91d6-962dce132dfd None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:43.238451 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 482/945] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:43.238595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938bbaf-67d6-484c-91d6-962dce132dfd None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:43.239453 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938bbaf-67d6-484c-91d6-962dce132dfd None admin] RBAC: Adding query filter params (type=compute) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:43.239672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2275e9-3ae8-4085-9a37-dd1d56ec3d98 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:43.240169 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938bbaf-67d6-484c-91d6-962dce132dfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:43.241699 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938bbaf-67d6-484c-91d6-962dce132dfd None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:43.255987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 464/946] 10.222.0.60 () {56 vars in 1112 bytes} [Thu Jul 21 20:39:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:43.291518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:43.371987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2275e9-3ae8-4085-9a37-dd1d56ec3d98 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:43.414303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c23b2c-8bca-42eb-adc1-6083fc41ffec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:43.417873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c23b2c-8bca-42eb-adc1-6083fc41ffec None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:43.417873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c23b2c-8bca-42eb-adc1-6083fc41ffec None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:43.417873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c23b2c-8bca-42eb-adc1-6083fc41ffec None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:43.417873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c23b2c-8bca-42eb-adc1-6083fc41ffec None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:43.417873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c23b2c-8bca-42eb-adc1-6083fc41ffec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:43.418265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c23b2c-8bca-42eb-adc1-6083fc41ffec None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:43.432370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 483/947] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:43.438547 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938bbaf-67d6-484c-91d6-962dce132dfd None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:43.554043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938bbaf-67d6-484c-91d6-962dce132dfd None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:43.591007 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d725060-4efb-4da9-8997-89dff6c1c346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:43.592163 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d725060-4efb-4da9-8997-89dff6c1c346 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:43.592591 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d725060-4efb-4da9-8997-89dff6c1c346 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:43.593029 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d725060-4efb-4da9-8997-89dff6c1c346 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:43.602474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d725060-4efb-4da9-8997-89dff6c1c346 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:43.604466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d725060-4efb-4da9-8997-89dff6c1c346 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-4d725060-4efb-4da9-8997-89dff6c1c346 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:43.620952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:43.622143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:43.623288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:43.623288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:43.623288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:43.623288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:43.623288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:43.623288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:43.623288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:43.623288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:43.623288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:39:43.623288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:43.625653 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 465/948] 10.222.0.60 () {56 vars in 1117 bytes} [Thu Jul 21 20:39:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:43.631539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c23b2c-8bca-42eb-adc1-6083fc41ffec None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:43.737812 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:43.743940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c23b2c-8bca-42eb-adc1-6083fc41ffec None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:43.781705 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752601c-7f9e-4fb2-b3ee-6c8c6b2dabcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:43.782393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752601c-7f9e-4fb2-b3ee-6c8c6b2dabcf None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:43.782678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752601c-7f9e-4fb2-b3ee-6c8c6b2dabcf None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:43.782957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752601c-7f9e-4fb2-b3ee-6c8c6b2dabcf None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:43.783841 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752601c-7f9e-4fb2-b3ee-6c8c6b2dabcf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:43.784156 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752601c-7f9e-4fb2-b3ee-6c8c6b2dabcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:43.786170 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752601c-7f9e-4fb2-b3ee-6c8c6b2dabcf None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:43.812183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 484/949] 10.222.0.60 () {56 vars in 1149 bytes} [Thu Jul 21 20:39:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:43.816925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d725060-4efb-4da9-8997-89dff6c1c346 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:43.826902 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:43.929302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dbcc9-ed60-4d63-a54d-0b9cbf078ed3 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:43.929600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dbcc9-ed60-4d63-a54d-0b9cbf078ed3 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:43.929888 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dbcc9-ed60-4d63-a54d-0b9cbf078ed3 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:43.931554 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 485/950] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:39:43.937054 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a79465-ea9f-4de3-8151-cc7b4086f837 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:43.937482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a79465-ea9f-4de3-8151-cc7b4086f837 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:43.937980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a79465-ea9f-4de3-8151-cc7b4086f837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:43.943610 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d725060-4efb-4da9-8997-89dff6c1c346 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:43.957711 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-56a79465-ea9f-4de3-8151-cc7b4086f837 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:43.959953 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-56a79465-ea9f-4de3-8151-cc7b4086f837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:43.992371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b5be2-8d82-4c8e-b17d-bd027ebd728f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:43.994176 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b5be2-8d82-4c8e-b17d-bd027ebd728f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:43.995765 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b5be2-8d82-4c8e-b17d-bd027ebd728f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:43.996607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b5be2-8d82-4c8e-b17d-bd027ebd728f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:44.017302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b5be2-8d82-4c8e-b17d-bd027ebd728f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:44.019197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b5be2-8d82-4c8e-b17d-bd027ebd728f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:44.043664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-4b6b5be2-8d82-4c8e-b17d-bd027ebd728f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:44.043664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:44.043664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:44.043664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:44.043664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:44.043664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:44.043664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:44.043664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:44.043664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:44.043664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:44.043664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:44.043664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:44.043664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:44.043664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:44.043664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:44.044919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:44.046073 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:44.046073 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:44.046073 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:44.046073 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:44.050351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 486/951] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:44.051875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 466/952] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:44.056274 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a79465-ea9f-4de3-8151-cc7b4086f837 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:44.084989 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b5be2-8d82-4c8e-b17d-bd027ebd728f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:44.153335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:44.170879 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:44.192248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a79465-ea9f-4de3-8151-cc7b4086f837 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:44.199700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b5be2-8d82-4c8e-b17d-bd027ebd728f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:44.232786 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf6f87-d91f-4c6e-b954-981502963216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:44.233911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf6f87-d91f-4c6e-b954-981502963216 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:44.234441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf6f87-d91f-4c6e-b954-981502963216 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:44.235013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf6f87-d91f-4c6e-b954-981502963216 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:44.235990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf6f87-d91f-4c6e-b954-981502963216 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:44.236656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf6f87-d91f-4c6e-b954-981502963216 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:44.238090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3320973-0eb8-4237-b198-5d70b263fd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:44.239308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3320973-0eb8-4237-b198-5d70b263fd1b None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:44.239592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3320973-0eb8-4237-b198-5d70b263fd1b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:44.239880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3320973-0eb8-4237-b198-5d70b263fd1b None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:44.240084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf6f87-d91f-4c6e-b954-981502963216 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:44.240674 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3320973-0eb8-4237-b198-5d70b263fd1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:44.242222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3320973-0eb8-4237-b198-5d70b263fd1b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:44.243839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: WARNING keystone.server.flask.common [None req-e3320973-0eb8-4237-b198-5d70b263fd1b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 20:39:44.259177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.sql.core [None req-e3320973-0eb8-4237-b198-5d70b263fd1b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 20:39:44.259177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 20:39:44.259177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [parameters: {'user_id': 'cdebcca37140469f8458c12309180593', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 20:39:44.259177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87060) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 20:39:44.260137 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 487/953] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:44.261232 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf6f87-d91f-4c6e-b954-981502963216 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:44.275409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-e3320973-0eb8-4237-b198-5d70b263fd1b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 20:39:44.275409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:44.275409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 20:39:44.275409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 20:39:44.275409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 20:39:44.275409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 20:39:44.275409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 20:39:44.275409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 20:39:44.275409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 20:39:44.275409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application conn.query(q) Jul 21 20:39:44.275409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 20:39:44.275409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 20:39:44.275409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 20:39:44.275409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application result.read() Jul 21 20:39:44.275409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 20:39:44.276388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application next(self.gen) Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application yield resource Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application next(self.gen) Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise self.value Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application session.commit() Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 20:39:44.277450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self.session.flush() Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application compat.raise_( Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 20:39:44.278328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 20:39:44.279212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application conn.query(q) Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application result.read() Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 20:39:44.280093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 20:39:44.284322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 20:39:44.284322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 20:39:44.284322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 20:39:44.284322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 20:39:44.284322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 20:39:44.284322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 20:39:44.284322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cdebcca37140469f8458c12309180593', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 20:39:44.284322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 20:39:44.284322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:44.284322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:44.284322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:44.284322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:44.284322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:44.284322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:44.284322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 21 20:39:44.285137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 21 20:39:44.287299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 21 20:39:44.287299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 21 20:39:44.287299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 20:39:44.287299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 20:39:44.287299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 20:39:44.287299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:44.287299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 467/954] 10.222.0.60 () {60 vars in 1131 bytes} [Thu Jul 21 20:39:44 2022] POST /identity/v3/users => generated 160 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:44.287299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3320973-0eb8-4237-b198-5d70b263fd1b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:44.391702 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf6f87-d91f-4c6e-b954-981502963216 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:44.428955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3320973-0eb8-4237-b198-5d70b263fd1b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:44.433789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc83eb-acc7-48c5-b986-c7950d5feb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:44.434882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc83eb-acc7-48c5-b986-c7950d5feb5b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:44.435386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc83eb-acc7-48c5-b986-c7950d5feb5b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:44.435931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc83eb-acc7-48c5-b986-c7950d5feb5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:44.444417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc83eb-acc7-48c5-b986-c7950d5feb5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:44.446448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc83eb-acc7-48c5-b986-c7950d5feb5b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:44.453976 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:44.466987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-01bc83eb-acc7-48c5-b986-c7950d5feb5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:44.466987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:44.466987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:44.466987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:44.466987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:44.466987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:44.466987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:44.466987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:44.466987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:44.466987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:44.466987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:44.466987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:44.466987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:44.466987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:44.466987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:44.468017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:44.468897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:44.468897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:44.468897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:44.468897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:44.468897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:44.468897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:44.468897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:44.468897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:44.472368 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 488/955] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:44.482622 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc83eb-acc7-48c5-b986-c7950d5feb5b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:44.491749 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3340400-c575-465b-ac6f-f21e46c6c587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:44.492583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3340400-c575-465b-ac6f-f21e46c6c587 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:44.492980 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3340400-c575-465b-ac6f-f21e46c6c587 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:44.493457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3340400-c575-465b-ac6f-f21e46c6c587 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:44.496573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:44.506727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3340400-c575-465b-ac6f-f21e46c6c587 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:44.508834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3340400-c575-465b-ac6f-f21e46c6c587 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-e3340400-c575-465b-ac6f-f21e46c6c587 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:44.524951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:44.525997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:44.526951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:44.526951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:44.526951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:44.526951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:44.526951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:44.526951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:44.526951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:44.526951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:44.526951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 20:39:44.526951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:44.530661 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 468/956] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:39:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:44.539351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3340400-c575-465b-ac6f-f21e46c6c587 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:44.630728 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc83eb-acc7-48c5-b986-c7950d5feb5b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:44.669759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3340400-c575-465b-ac6f-f21e46c6c587 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:44.674046 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29468e45-cbb5-4481-a528-b34d2e8116d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:44.674743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29468e45-cbb5-4481-a528-b34d2e8116d3 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:44.675030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29468e45-cbb5-4481-a528-b34d2e8116d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:44.675309 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29468e45-cbb5-4481-a528-b34d2e8116d3 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:44.676708 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29468e45-cbb5-4481-a528-b34d2e8116d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:44.677028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29468e45-cbb5-4481-a528-b34d2e8116d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:44.678487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29468e45-cbb5-4481-a528-b34d2e8116d3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:44.699809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 489/957] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:44.702397 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29468e45-cbb5-4481-a528-b34d2e8116d3 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:44.721535 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7b798-202a-4086-923a-ea1f37d9a75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:44.722659 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7b798-202a-4086-923a-ea1f37d9a75b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:44.723859 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7b798-202a-4086-923a-ea1f37d9a75b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:44.724584 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7b798-202a-4086-923a-ea1f37d9a75b None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:44.725690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7b798-202a-4086-923a-ea1f37d9a75b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:44.726222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7b798-202a-4086-923a-ea1f37d9a75b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:44.728262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7b798-202a-4086-923a-ea1f37d9a75b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:44.773412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 469/958] 10.222.0.60 () {56 vars in 1101 bytes} [Thu Jul 21 20:39:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:44.782048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:44.830618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29468e45-cbb5-4481-a528-b34d2e8116d3 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:44.834782 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112307b9-6c75-40a5-aee0-fe95b745c99c None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:44.835091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112307b9-6c75-40a5-aee0-fe95b745c99c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:44.835349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112307b9-6c75-40a5-aee0-fe95b745c99c None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:44.838361 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 470/959] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:44.845216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd4e54-cf1e-4bd8-8e76-b7f4e2287fdf None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:44.845670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd4e54-cf1e-4bd8-8e76-b7f4e2287fdf None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:44.846063 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd4e54-cf1e-4bd8-8e76-b7f4e2287fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:44.867941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-88fd4e54-cf1e-4bd8-8e76-b7f4e2287fdf None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:44.869971 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-88fd4e54-cf1e-4bd8-8e76-b7f4e2287fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:44.888042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b1f95-4a7c-4359-8adb-ab91afece0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:44.889102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b1f95-4a7c-4359-8adb-ab91afece0d7 None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:44.889455 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b1f95-4a7c-4359-8adb-ab91afece0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:44.889847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b1f95-4a7c-4359-8adb-ab91afece0d7 None admin] PATH_INFO: `/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/f0beb65e5dc7469eb17a98dae1f09a0e/roles/952e60f3db5044afa8b3043155a1ccca` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:44.892606 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b1f95-4a7c-4359-8adb-ab91afece0d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc61fe15894891a9054911321f4ac3, user_id=f0beb65e5dc7469eb17a98dae1f09a0e, role_id=952e60f3db5044afa8b3043155a1ccca)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:44.894278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b1f95-4a7c-4359-8adb-ab91afece0d7 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:44.916004 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:44.917697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 490/960] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:39:44 2022] PUT /identity/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/f0beb65e5dc7469eb17a98dae1f09a0e/roles/952e60f3db5044afa8b3043155a1ccca => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:39:44.981009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 471/961] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:45.012893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b1f95-4a7c-4359-8adb-ab91afece0d7 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:45.134685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b1f95-4a7c-4359-8adb-ab91afece0d7 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:45.162633 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:45.179674 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322607f8-f5bb-476c-8198-57e040bef648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:45.180532 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322607f8-f5bb-476c-8198-57e040bef648 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:45.180887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322607f8-f5bb-476c-8198-57e040bef648 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:45.181088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322607f8-f5bb-476c-8198-57e040bef648 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:45.181727 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322607f8-f5bb-476c-8198-57e040bef648 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:45.183326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322607f8-f5bb-476c-8198-57e040bef648 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-322607f8-f5bb-476c-8198-57e040bef648 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:45.196809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:45.197943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:45.198962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:45.198962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:45.198962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:45.198962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:45.198962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:45.198962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:45.198962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:45.198962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:45.198962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:45.198962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:45.198962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:45.198962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:45.198962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:39:45.198962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:45.198962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 491/962] 10.222.0.60 () {56 vars in 1126 bytes} [Thu Jul 21 20:39:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:45.204310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd4e54-cf1e-4bd8-8e76-b7f4e2287fdf None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:45.322836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd4e54-cf1e-4bd8-8e76-b7f4e2287fdf None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:45.360104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90858b-0c64-4fb0-a7a4-f47ec5b4f0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:45.360786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90858b-0c64-4fb0-a7a4-f47ec5b4f0cf None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:45.361068 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90858b-0c64-4fb0-a7a4-f47ec5b4f0cf None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:45.361348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90858b-0c64-4fb0-a7a4-f47ec5b4f0cf None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:45.362065 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90858b-0c64-4fb0-a7a4-f47ec5b4f0cf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:45.362372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90858b-0c64-4fb0-a7a4-f47ec5b4f0cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:45.364557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90858b-0c64-4fb0-a7a4-f47ec5b4f0cf None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:45.369836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 472/963] 10.222.0.60 () {56 vars in 1122 bytes} [Thu Jul 21 20:39:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:45.375281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322607f8-f5bb-476c-8198-57e040bef648 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:45.458285 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522e2b2-6f60-4c5f-b790-e5b01e1a9cbc None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:45.458662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522e2b2-6f60-4c5f-b790-e5b01e1a9cbc None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:45.459018 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522e2b2-6f60-4c5f-b790-e5b01e1a9cbc None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:45.460711 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 473/964] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:45.466581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96daa4cb-8a95-42ad-97f8-caa55d481fc1 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:45.467020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96daa4cb-8a95-42ad-97f8-caa55d481fc1 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:45.467304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96daa4cb-8a95-42ad-97f8-caa55d481fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:45.490855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-96daa4cb-8a95-42ad-97f8-caa55d481fc1 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:45.492094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-96daa4cb-8a95-42ad-97f8-caa55d481fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:45.510705 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322607f8-f5bb-476c-8198-57e040bef648 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:45.525285 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:45.546523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e4867-cd35-4b59-8393-8aecbe2d934e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:45.547190 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e4867-cd35-4b59-8393-8aecbe2d934e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:45.547472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e4867-cd35-4b59-8393-8aecbe2d934e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:45.547966 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e4867-cd35-4b59-8393-8aecbe2d934e None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:45.548674 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e4867-cd35-4b59-8393-8aecbe2d934e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:45.549135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e4867-cd35-4b59-8393-8aecbe2d934e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:45.550507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e4867-cd35-4b59-8393-8aecbe2d934e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:45.561773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 492/965] 10.222.0.60 () {56 vars in 1122 bytes} [Thu Jul 21 20:39:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:45.563199 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e4867-cd35-4b59-8393-8aecbe2d934e None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:45.587098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 474/966] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:45.602262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96daa4cb-8a95-42ad-97f8-caa55d481fc1 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:45.622775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:45.693792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e4867-cd35-4b59-8393-8aecbe2d934e None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:45.725922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96daa4cb-8a95-42ad-97f8-caa55d481fc1 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:45.737646 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6461548-f207-4c8f-9d6c-a3e4818c0b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:45.738318 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6461548-f207-4c8f-9d6c-a3e4818c0b57 None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:45.739401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6461548-f207-4c8f-9d6c-a3e4818c0b57 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:45.739694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6461548-f207-4c8f-9d6c-a3e4818c0b57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:45.740446 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6461548-f207-4c8f-9d6c-a3e4818c0b57 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:45.741840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6461548-f207-4c8f-9d6c-a3e4818c0b57 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:45.764345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 493/967] 10.222.0.60 () {60 vars in 1144 bytes} [Thu Jul 21 20:39:45 2022] POST /identity/v3/endpoints => generated 349 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:39:45.781792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ee545-44f4-4baa-b40d-dbcb12f69008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:45.782469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ee545-44f4-4baa-b40d-dbcb12f69008 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:45.782747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ee545-44f4-4baa-b40d-dbcb12f69008 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:45.783012 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ee545-44f4-4baa-b40d-dbcb12f69008 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:45.783784 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ee545-44f4-4baa-b40d-dbcb12f69008 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:45.785210 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ee545-44f4-4baa-b40d-dbcb12f69008 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:45.805631 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 475/968] 10.222.0.60 () {60 vars in 1141 bytes} [Thu Jul 21 20:39:45 2022] POST /identity/v3/services => generated 242 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:39:47.101312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7010ffa-fba8-4345-8f32-435eca392749 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:47.101873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7010ffa-fba8-4345-8f32-435eca392749 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:47.101873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7010ffa-fba8-4345-8f32-435eca392749 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:47.103296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 494/969] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:47.108909 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb27aca-1daf-43a0-9f0a-fbc980176801 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:47.109328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb27aca-1daf-43a0-9f0a-fbc980176801 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:47.109500 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb27aca-1daf-43a0-9f0a-fbc980176801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:47.126750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-ecb27aca-1daf-43a0-9f0a-fbc980176801 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:47.127985 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ecb27aca-1daf-43a0-9f0a-fbc980176801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:47.142987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:47.208740 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 476/970] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:47.233313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7010ffa-fba8-4345-8f32-435eca392749 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:47.342750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7010ffa-fba8-4345-8f32-435eca392749 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:47.353109 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:47.383038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4e32c-bd39-44e9-b200-cf1a3c578df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:47.383916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4e32c-bd39-44e9-b200-cf1a3c578df0 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:47.384240 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4e32c-bd39-44e9-b200-cf1a3c578df0 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:47.384557 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4e32c-bd39-44e9-b200-cf1a3c578df0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:47.403773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4e32c-bd39-44e9-b200-cf1a3c578df0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-0ed4e32c-bd39-44e9-b200-cf1a3c578df0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:47.407013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:47.412296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:47.413371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:47.413371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:47.413371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:47.413371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:47.413371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:47.413371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:47.413371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:47.413371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:47.413371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:47.413371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:47.413371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:39:47.413371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:47.413371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 495/971] 10.222.0.60 () {56 vars in 1120 bytes} [Thu Jul 21 20:39:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:47.421062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb27aca-1daf-43a0-9f0a-fbc980176801 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:47.536688 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb27aca-1daf-43a0-9f0a-fbc980176801 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:47.593941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a68af9-28fb-4456-a20c-e261c027eab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:47.594622 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a68af9-28fb-4456-a20c-e261c027eab4 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:47.594908 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a68af9-28fb-4456-a20c-e261c027eab4 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:47.595185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a68af9-28fb-4456-a20c-e261c027eab4 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:47.596002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a68af9-28fb-4456-a20c-e261c027eab4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:47.596317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a68af9-28fb-4456-a20c-e261c027eab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:47.597750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a68af9-28fb-4456-a20c-e261c027eab4 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:47.607134 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 477/972] 10.222.0.60 () {56 vars in 1115 bytes} [Thu Jul 21 20:39:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:47.612701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4e32c-bd39-44e9-b200-cf1a3c578df0 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:47.722495 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4e32c-bd39-44e9-b200-cf1a3c578df0 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:47.760359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0a3f4-4b41-4ae4-8923-8b32d4cc824f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:47.761168 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0a3f4-4b41-4ae4-8923-8b32d4cc824f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:47.761455 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0a3f4-4b41-4ae4-8923-8b32d4cc824f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:47.761734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0a3f4-4b41-4ae4-8923-8b32d4cc824f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:47.773726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0a3f4-4b41-4ae4-8923-8b32d4cc824f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:47.775201 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0a3f4-4b41-4ae4-8923-8b32d4cc824f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:47.796893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-aed0a3f4-4b41-4ae4-8923-8b32d4cc824f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:47.796893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:47.796893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:47.796893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:47.796893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:47.796893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:47.796893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:47.796893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:47.796893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:47.796893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:47.796893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:47.796893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:47.796893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:47.796893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:47.796893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:47.797870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:47.798847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:47.798847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:47.798847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:47.798847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:47.800033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 496/973] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:47.804486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a68af9-28fb-4456-a20c-e261c027eab4 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:47.814278 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:47.896186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a29a9-6a38-419d-a0c0-50acd3f0f642 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:47.896391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a29a9-6a38-419d-a0c0-50acd3f0f642 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:47.897167 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a29a9-6a38-419d-a0c0-50acd3f0f642 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:47.898921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 497/974] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:39:47.904525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635e000-75eb-4805-a49e-a7aed6efb82d None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:47.904800 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635e000-75eb-4805-a49e-a7aed6efb82d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:47.905077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635e000-75eb-4805-a49e-a7aed6efb82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:47.922685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-1635e000-75eb-4805-a49e-a7aed6efb82d None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:47.923084 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a68af9-28fb-4456-a20c-e261c027eab4 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:47.923931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1635e000-75eb-4805-a49e-a7aed6efb82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:47.940049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:47.958620 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8acea-fd99-4345-bba1-d330713f0ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:47.959520 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8acea-fd99-4345-bba1-d330713f0ad6 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:47.959828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8acea-fd99-4345-bba1-d330713f0ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:47.960100 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8acea-fd99-4345-bba1-d330713f0ad6 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:47.960821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8acea-fd99-4345-bba1-d330713f0ad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:47.961119 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8acea-fd99-4345-bba1-d330713f0ad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:47.962468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8acea-fd99-4345-bba1-d330713f0ad6 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:47.974840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 478/975] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:47.976140 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d7680-5689-4ea5-b27c-b621a5227b30 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:47.976424 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d7680-5689-4ea5-b27c-b621a5227b30 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:47.976694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d7680-5689-4ea5-b27c-b621a5227b30 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:47.977907 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 479/976] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:47.980431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d7680-5689-4ea5-b27c-b621a5227b30 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:48.002195 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 498/977] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:48.003696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cabd1f-0e8b-4bf1-81f8-8946afc0e752 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:48.004062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cabd1f-0e8b-4bf1-81f8-8946afc0e752 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:48.004338 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cabd1f-0e8b-4bf1-81f8-8946afc0e752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:48.024843 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d4cabd1f-0e8b-4bf1-81f8-8946afc0e752 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:48.026128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d4cabd1f-0e8b-4bf1-81f8-8946afc0e752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:48.052308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:48.095035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d7680-5689-4ea5-b27c-b621a5227b30 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:48.100356 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 499/978] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:48.101212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cabd1f-0e8b-4bf1-81f8-8946afc0e752 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:48.132826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08851e13-f18a-473d-a215-b8ce607a02e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:48.133643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08851e13-f18a-473d-a215-b8ce607a02e9 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:48.133930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08851e13-f18a-473d-a215-b8ce607a02e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:48.134206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08851e13-f18a-473d-a215-b8ce607a02e9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:48.142233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08851e13-f18a-473d-a215-b8ce607a02e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:48.143698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08851e13-f18a-473d-a215-b8ce607a02e9 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-08851e13-f18a-473d-a215-b8ce607a02e9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:48.161177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:48.162351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:48.163437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:48.163437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:48.163437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:48.163437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:48.163437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:48.163437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:48.163437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:48.163437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:48.163437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 20:39:48.163437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:48.163437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 480/979] 10.222.0.60 () {56 vars in 1117 bytes} [Thu Jul 21 20:39:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:48.164871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08851e13-f18a-473d-a215-b8ce607a02e9 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:48.189841 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:48.214755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cabd1f-0e8b-4bf1-81f8-8946afc0e752 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:48.250299 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797494be-7a5e-4bda-b198-44349bf2a881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:48.251376 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797494be-7a5e-4bda-b198-44349bf2a881 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:48.251654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797494be-7a5e-4bda-b198-44349bf2a881 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:48.251961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797494be-7a5e-4bda-b198-44349bf2a881 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:48.252618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797494be-7a5e-4bda-b198-44349bf2a881 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:48.254561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797494be-7a5e-4bda-b198-44349bf2a881 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:48.271953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08851e13-f18a-473d-a215-b8ce607a02e9 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-797494be-7a5e-4bda-b198-44349bf2a881 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:48.272529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:48.273521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:48.274337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:48.274337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:48.274337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:48.274337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:48.274337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:48.274337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:48.274337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:48.274337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:48.274337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:48.274337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:48.274337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:48.274337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:48.274337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:39:48.274337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:48.275054 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 500/980] 10.222.0.60 () {56 vars in 1126 bytes} [Thu Jul 21 20:39:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:48.275851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797494be-7a5e-4bda-b198-44349bf2a881 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:48.303805 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:48.313371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8aa8a-b61f-4513-93b5-a03da7dff964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:48.314237 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8aa8a-b61f-4513-93b5-a03da7dff964 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:48.314518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8aa8a-b61f-4513-93b5-a03da7dff964 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:48.314799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8aa8a-b61f-4513-93b5-a03da7dff964 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:48.315470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8aa8a-b61f-4513-93b5-a03da7dff964 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:48.317453 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8aa8a-b61f-4513-93b5-a03da7dff964 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-72a8aa8a-b61f-4513-93b5-a03da7dff964 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:48.329567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:48.330700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:48.331527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:48.331527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:48.331527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:48.331527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:48.331527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:48.331527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:48.331527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:48.331527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:48.331527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:48.331527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:48.331527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:48.331527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:48.331527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:39:48.331527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:48.331527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 481/981] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:48.333258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8aa8a-b61f-4513-93b5-a03da7dff964 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:48.383069 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797494be-7a5e-4bda-b198-44349bf2a881 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:48.423175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207200ec-2afe-4bb3-ab96-5935485caeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:48.423797 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207200ec-2afe-4bb3-ab96-5935485caeca None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:48.424082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207200ec-2afe-4bb3-ab96-5935485caeca None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:48.424358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207200ec-2afe-4bb3-ab96-5935485caeca None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:48.425197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207200ec-2afe-4bb3-ab96-5935485caeca None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:48.425504 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207200ec-2afe-4bb3-ab96-5935485caeca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:48.427768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207200ec-2afe-4bb3-ab96-5935485caeca None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:48.440548 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8aa8a-b61f-4513-93b5-a03da7dff964 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:48.447558 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 501/982] 10.222.0.60 () {56 vars in 1149 bytes} [Thu Jul 21 20:39:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:48.448434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207200ec-2afe-4bb3-ab96-5935485caeca None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:48.451197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:48.478968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648fced-088d-4528-a1a9-8fb583cf0a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:48.479925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648fced-088d-4528-a1a9-8fb583cf0a4b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:48.480218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648fced-088d-4528-a1a9-8fb583cf0a4b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:48.480749 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648fced-088d-4528-a1a9-8fb583cf0a4b None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:48.481240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648fced-088d-4528-a1a9-8fb583cf0a4b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:48.481548 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648fced-088d-4528-a1a9-8fb583cf0a4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:48.482965 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648fced-088d-4528-a1a9-8fb583cf0a4b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:48.488104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 482/983] 10.222.0.60 () {56 vars in 1122 bytes} [Thu Jul 21 20:39:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:48.488987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648fced-088d-4528-a1a9-8fb583cf0a4b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:48.552985 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:48.562453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207200ec-2afe-4bb3-ab96-5935485caeca None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:48.600717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9784205-777f-4e69-add8-ef97775201db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:48.601066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648fced-088d-4528-a1a9-8fb583cf0a4b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:48.601436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9784205-777f-4e69-add8-ef97775201db None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:48.601698 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9784205-777f-4e69-add8-ef97775201db None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:48.601971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9784205-777f-4e69-add8-ef97775201db None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:48.602680 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9784205-777f-4e69-add8-ef97775201db None admin] RBAC: Adding query filter params (name=compute) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:48.602984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9784205-777f-4e69-add8-ef97775201db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:48.604647 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9784205-777f-4e69-add8-ef97775201db None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:48.610162 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 502/984] 10.222.0.60 () {56 vars in 1112 bytes} [Thu Jul 21 20:39:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:48.610972 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9784205-777f-4e69-add8-ef97775201db None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:48.640349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b82681-1340-4c6c-8335-a486f37a0cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:48.641009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b82681-1340-4c6c-8335-a486f37a0cdc None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:48.641299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b82681-1340-4c6c-8335-a486f37a0cdc None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:48.641579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b82681-1340-4c6c-8335-a486f37a0cdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:48.680439 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:48.711644 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b82681-1340-4c6c-8335-a486f37a0cdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:48.713122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b82681-1340-4c6c-8335-a486f37a0cdc None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:48.717354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9784205-777f-4e69-add8-ef97775201db None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:48.756126 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28df4dd-a0f8-4942-8e71-e2ce3782ea87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:48.756800 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28df4dd-a0f8-4942-8e71-e2ce3782ea87 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:48.757547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28df4dd-a0f8-4942-8e71-e2ce3782ea87 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:48.757834 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28df4dd-a0f8-4942-8e71-e2ce3782ea87 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:48.758555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28df4dd-a0f8-4942-8e71-e2ce3782ea87 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:48.758871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28df4dd-a0f8-4942-8e71-e2ce3782ea87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:48.760643 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28df4dd-a0f8-4942-8e71-e2ce3782ea87 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:48.765956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 503/985] 10.222.0.60 () {56 vars in 1122 bytes} [Thu Jul 21 20:39:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:48.766777 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28df4dd-a0f8-4942-8e71-e2ce3782ea87 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:48.778277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:48.826394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 483/986] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:39:48 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:39:48.827236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b82681-1340-4c6c-8335-a486f37a0cdc None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:48.877862 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28df4dd-a0f8-4942-8e71-e2ce3782ea87 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:48.911004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:48.914049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0a5ac-7339-4907-b732-532c27fa4473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:48.914926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0a5ac-7339-4907-b732-532c27fa4473 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:48.915368 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0a5ac-7339-4907-b732-532c27fa4473 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:48.915653 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0a5ac-7339-4907-b732-532c27fa4473 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:48.916644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0a5ac-7339-4907-b732-532c27fa4473 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:48.916952 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0a5ac-7339-4907-b732-532c27fa4473 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:48.918335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0a5ac-7339-4907-b732-532c27fa4473 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:48.923544 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 504/987] 10.222.0.60 () {56 vars in 1112 bytes} [Thu Jul 21 20:39:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:48.924401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0a5ac-7339-4907-b732-532c27fa4473 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:48.937585 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b82681-1340-4c6c-8335-a486f37a0cdc None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:48.975047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291f18d-9e35-4ecc-9c28-be43aab7f5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:48.975740 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291f18d-9e35-4ecc-9c28-be43aab7f5e8 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:48.976027 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291f18d-9e35-4ecc-9c28-be43aab7f5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:48.976303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291f18d-9e35-4ecc-9c28-be43aab7f5e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:48.977117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291f18d-9e35-4ecc-9c28-be43aab7f5e8 None admin] RBAC: Adding query filter params (interface=admin, service_id=b30ddcac40b04bc083b45252ed10a423, region_id=RegionOne) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:48.977432 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291f18d-9e35-4ecc-9c28-be43aab7f5e8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:48.978816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291f18d-9e35-4ecc-9c28-be43aab7f5e8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:48.984883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 484/988] 10.222.0.60 () {56 vars in 1249 bytes} [Thu Jul 21 20:39:48 2022] GET /identity/v3/endpoints?service_id=b30ddcac40b04bc083b45252ed10a423&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:48.985697 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291f18d-9e35-4ecc-9c28-be43aab7f5e8 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:49.006218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:49.040950 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0a5ac-7339-4907-b732-532c27fa4473 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:49.081198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4e832-7e8c-4d37-a4a3-9e9146c4e32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:49.081997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4e832-7e8c-4d37-a4a3-9e9146c4e32d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:49.082850 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4e832-7e8c-4d37-a4a3-9e9146c4e32d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:49.083122 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4e832-7e8c-4d37-a4a3-9e9146c4e32d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:49.092935 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291f18d-9e35-4ecc-9c28-be43aab7f5e8 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:49.093965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4e832-7e8c-4d37-a4a3-9e9146c4e32d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:49.095432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4e832-7e8c-4d37-a4a3-9e9146c4e32d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:49.115094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-4bd4e832-7e8c-4d37-a4a3-9e9146c4e32d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:49.115094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:49.115094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:49.115094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:49.115094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:49.115094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:49.115094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:49.115094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:49.115094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:49.115094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:49.115094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:49.115094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:49.115094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:49.115094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:49.115094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:49.116497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:49.117426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:49.117426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:49.117426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:49.117426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:49.117426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 505/989] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:49.117426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4e832-7e8c-4d37-a4a3-9e9146c4e32d None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:49.144078 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59e658-de20-4315-bcfd-1ba65c0094a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:49.145564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59e658-de20-4315-bcfd-1ba65c0094a9 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:49.145825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59e658-de20-4315-bcfd-1ba65c0094a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:49.146223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59e658-de20-4315-bcfd-1ba65c0094a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:49.147038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59e658-de20-4315-bcfd-1ba65c0094a9 None admin] RBAC: Adding query filter params (interface=public, service_id=50a4f7400bc145419c2917bc7295c43c, region_id=RegionOne) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:49.147350 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59e658-de20-4315-bcfd-1ba65c0094a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:49.148764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59e658-de20-4315-bcfd-1ba65c0094a9 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:49.154215 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 485/990] 10.222.0.60 () {56 vars in 1251 bytes} [Thu Jul 21 20:39:48 2022] GET /identity/v3/endpoints?service_id=50a4f7400bc145419c2917bc7295c43c&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:49.155040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59e658-de20-4315-bcfd-1ba65c0094a9 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:49.173863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:49.230189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4e832-7e8c-4d37-a4a3-9e9146c4e32d None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:49.244598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:49.265363 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59e658-de20-4315-bcfd-1ba65c0094a9 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:49.267732 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1e2d3-e798-4127-ada1-e97d37951371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:49.268404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1e2d3-e798-4127-ada1-e97d37951371 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:49.268687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1e2d3-e798-4127-ada1-e97d37951371 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:49.268959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1e2d3-e798-4127-ada1-e97d37951371 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:49.269997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1e2d3-e798-4127-ada1-e97d37951371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:49.271358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1e2d3-e798-4127-ada1-e97d37951371 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:49.278003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 506/991] 10.222.0.60 () {56 vars in 1087 bytes} [Thu Jul 21 20:39:49 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:49.278874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1e2d3-e798-4127-ada1-e97d37951371 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:49.302486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228ee42-6441-4ff0-a1a6-9152b41aff59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:49.303509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228ee42-6441-4ff0-a1a6-9152b41aff59 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:49.303839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228ee42-6441-4ff0-a1a6-9152b41aff59 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:49.304116 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228ee42-6441-4ff0-a1a6-9152b41aff59 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:49.304860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228ee42-6441-4ff0-a1a6-9152b41aff59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:49.305166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228ee42-6441-4ff0-a1a6-9152b41aff59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:49.306607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228ee42-6441-4ff0-a1a6-9152b41aff59 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:49.320969 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 486/992] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:49.326588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228ee42-6441-4ff0-a1a6-9152b41aff59 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:49.439959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1e2d3-e798-4127-ada1-e97d37951371 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:49.453501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:49.456568 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228ee42-6441-4ff0-a1a6-9152b41aff59 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:49.486548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa748af8-12cc-4ef7-b616-08f20c7f4fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:49.487411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa748af8-12cc-4ef7-b616-08f20c7f4fed None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:49.487704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa748af8-12cc-4ef7-b616-08f20c7f4fed None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:49.488013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa748af8-12cc-4ef7-b616-08f20c7f4fed None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:49.489515 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa748af8-12cc-4ef7-b616-08f20c7f4fed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:49.491798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa748af8-12cc-4ef7-b616-08f20c7f4fed None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:49.495474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406ccbc-479a-4e33-92db-ef7253929374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:49.496303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406ccbc-479a-4e33-92db-ef7253929374 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:49.496589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406ccbc-479a-4e33-92db-ef7253929374 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:49.496864 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406ccbc-479a-4e33-92db-ef7253929374 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:49.507924 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 507/993] 10.222.0.60 () {56 vars in 1087 bytes} [Thu Jul 21 20:39:49 2022] GET /identity/v3/services => generated 1727 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:49.508132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406ccbc-479a-4e33-92db-ef7253929374 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:49.508132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406ccbc-479a-4e33-92db-ef7253929374 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:49.524850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-6406ccbc-479a-4e33-92db-ef7253929374 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:49.524850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:49.524850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:49.524850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:49.524850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:49.524850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:49.524850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:49.524850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:49.524850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:49.524850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:49.524850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:49.524850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:49.524850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:49.524850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:49.524850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:49.525830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:49.526707 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:49.526707 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:49.526707 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:49.526707 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:49.526707 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:49.526707 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:49.526707 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:49.526707 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:49.528044 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 487/994] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:49.534602 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa748af8-12cc-4ef7-b616-08f20c7f4fed None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:49.654796 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa748af8-12cc-4ef7-b616-08f20c7f4fed None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:49.705630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f61b9-5f51-4e9f-85e2-fc941cc30dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:49.706710 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f61b9-5f51-4e9f-85e2-fc941cc30dc5 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:49.707789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f61b9-5f51-4e9f-85e2-fc941cc30dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:49.708256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f61b9-5f51-4e9f-85e2-fc941cc30dc5 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:49.709697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f61b9-5f51-4e9f-85e2-fc941cc30dc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:49.710220 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f61b9-5f51-4e9f-85e2-fc941cc30dc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:49.712893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f61b9-5f51-4e9f-85e2-fc941cc30dc5 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:49.734521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 508/995] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:49.740801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406ccbc-479a-4e33-92db-ef7253929374 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:49.744964 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:49.869600 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406ccbc-479a-4e33-92db-ef7253929374 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:49.915005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e35a5-47e0-4003-874f-8dc5bfdca06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:49.915672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e35a5-47e0-4003-874f-8dc5bfdca06e None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:49.915987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e35a5-47e0-4003-874f-8dc5bfdca06e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:49.916266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e35a5-47e0-4003-874f-8dc5bfdca06e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:49.917113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e35a5-47e0-4003-874f-8dc5bfdca06e None admin] RBAC: Adding query filter params (role.id=952e60f3db5044afa8b3043155a1ccca, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=f0beb65e5dc7469eb17a98dae1f09a0e) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:49.917427 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e35a5-47e0-4003-874f-8dc5bfdca06e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:49.918878 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e35a5-47e0-4003-874f-8dc5bfdca06e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:49.925311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 488/996] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:39:49 2022] GET /identity/v3/role_assignments?user.id=f0beb65e5dc7469eb17a98dae1f09a0e&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=952e60f3db5044afa8b3043155a1ccca => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:51.410784 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ed0b7-4fdd-4cb1-a3be-395402886c40 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:51.411338 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ed0b7-4fdd-4cb1-a3be-395402886c40 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:51.411338 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ed0b7-4fdd-4cb1-a3be-395402886c40 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:51.412920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 509/997] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:51.418688 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e4033-e070-4fa7-8b05-4e000cf7dc8a None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:51.419558 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e4033-e070-4fa7-8b05-4e000cf7dc8a None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:51.419888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e4033-e070-4fa7-8b05-4e000cf7dc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:51.437663 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-7a9e4033-e070-4fa7-8b05-4e000cf7dc8a None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:51.439054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7a9e4033-e070-4fa7-8b05-4e000cf7dc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:51.505251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:51.516625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 489/998] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:51.540581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ed0b7-4fdd-4cb1-a3be-395402886c40 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:51.579507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:51.633032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f8da7-a3a7-459a-b670-26e4b158a746 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:51.633382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f8da7-a3a7-459a-b670-26e4b158a746 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:51.633583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f8da7-a3a7-459a-b670-26e4b158a746 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:51.635142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 490/999] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:39:51.640095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa021ec-de77-44bf-a79f-177fd1cb93e4 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:51.640379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa021ec-de77-44bf-a79f-177fd1cb93e4 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:51.640654 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa021ec-de77-44bf-a79f-177fd1cb93e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:51.658239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-eaa021ec-de77-44bf-a79f-177fd1cb93e4 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:51.659348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-eaa021ec-de77-44bf-a79f-177fd1cb93e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:51.663183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ed0b7-4fdd-4cb1-a3be-395402886c40 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:51.707491 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f67ed-7559-4a3f-a651-efe1404c5eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:51.708873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f67ed-7559-4a3f-a651-efe1404c5eba None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:51.709351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f67ed-7559-4a3f-a651-efe1404c5eba None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:51.709820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f67ed-7559-4a3f-a651-efe1404c5eba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:51.710958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f67ed-7559-4a3f-a651-efe1404c5eba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:51.713386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f67ed-7559-4a3f-a651-efe1404c5eba None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-5b2f67ed-7559-4a3f-a651-efe1404c5eba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:51.736066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:51.738407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:51.739476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:51.739476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:51.739476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:51.739476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:51.739476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:51.739476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:51.739476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:51.739476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:51.739476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:51.739476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:51.739476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:51.739476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:51.739476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 20:39:51.739476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:51.739476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 510/1000] 10.222.0.60 () {56 vars in 1126 bytes} [Thu Jul 21 20:39:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:51.745266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f67ed-7559-4a3f-a651-efe1404c5eba None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:51.761841 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 491/1001] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:51.785776 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa021ec-de77-44bf-a79f-177fd1cb93e4 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:51.861802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:51.866815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f67ed-7559-4a3f-a651-efe1404c5eba None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:51.904686 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa021ec-de77-44bf-a79f-177fd1cb93e4 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:51.910678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb42b0-07d7-48ea-a560-373489668a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:51.911343 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb42b0-07d7-48ea-a560-373489668a8b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:51.911631 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb42b0-07d7-48ea-a560-373489668a8b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:51.911941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb42b0-07d7-48ea-a560-373489668a8b None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:51.912650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb42b0-07d7-48ea-a560-373489668a8b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:51.912952 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb42b0-07d7-48ea-a560-373489668a8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:51.914339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb42b0-07d7-48ea-a560-373489668a8b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:51.919860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 511/1002] 10.222.0.60 () {56 vars in 1122 bytes} [Thu Jul 21 20:39:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:51.925469 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb42b0-07d7-48ea-a560-373489668a8b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:51.933617 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:51.945870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbca6ec-c4e9-4120-afa4-8daca6b226e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:51.947197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbca6ec-c4e9-4120-afa4-8daca6b226e4 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:51.947683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbca6ec-c4e9-4120-afa4-8daca6b226e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:51.948118 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbca6ec-c4e9-4120-afa4-8daca6b226e4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:51.949063 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbca6ec-c4e9-4120-afa4-8daca6b226e4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:51.950451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbca6ec-c4e9-4120-afa4-8daca6b226e4 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-0fbca6ec-c4e9-4120-afa4-8daca6b226e4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:51.963124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:51.965037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:51.966641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:51.966641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:51.966641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:51.966641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:51.966641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:51.966641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:51.966641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:51.966641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:51.966641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:51.966641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:51.966641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:51.966641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:51.966641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 20:39:51.966641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:51.966641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 492/1003] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:51.971118 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbca6ec-c4e9-4120-afa4-8daca6b226e4 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:52.040276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb42b0-07d7-48ea-a560-373489668a8b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:52.080777 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392608c-d025-440c-89bd-96fd3e4ee9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:52.081344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbca6ec-c4e9-4120-afa4-8daca6b226e4 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:52.082643 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392608c-d025-440c-89bd-96fd3e4ee9bd None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.083358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392608c-d025-440c-89bd-96fd3e4ee9bd None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.083934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392608c-d025-440c-89bd-96fd3e4ee9bd None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.084997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392608c-d025-440c-89bd-96fd3e4ee9bd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:52.085454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392608c-d025-440c-89bd-96fd3e4ee9bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:52.086844 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392608c-d025-440c-89bd-96fd3e4ee9bd None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:52.097157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 512/1004] 10.222.0.60 () {56 vars in 1122 bytes} [Thu Jul 21 20:39:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:52.097157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875f8fa-41fa-4570-90be-128314c896c7 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.097157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875f8fa-41fa-4570-90be-128314c896c7 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.097157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875f8fa-41fa-4570-90be-128314c896c7 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.097157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 513/1005] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:52.099658 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875f8fa-41fa-4570-90be-128314c896c7 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:52.127397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afd3ad-3ef2-4da4-bb40-88bde6f50b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:52.128221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:52.128404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afd3ad-3ef2-4da4-bb40-88bde6f50b79 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.128553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afd3ad-3ef2-4da4-bb40-88bde6f50b79 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.129374 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afd3ad-3ef2-4da4-bb40-88bde6f50b79 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.130485 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afd3ad-3ef2-4da4-bb40-88bde6f50b79 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:52.130799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afd3ad-3ef2-4da4-bb40-88bde6f50b79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:52.132243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afd3ad-3ef2-4da4-bb40-88bde6f50b79 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:52.137426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 493/1006] 10.222.0.60 () {56 vars in 1112 bytes} [Thu Jul 21 20:39:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:52.140067 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5bee1-0736-4b77-b91d-66b0bb9bfec4 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.140067 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5bee1-0736-4b77-b91d-66b0bb9bfec4 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.140067 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5bee1-0736-4b77-b91d-66b0bb9bfec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.156599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-16c5bee1-0736-4b77-b91d-66b0bb9bfec4 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:52.157811 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-16c5bee1-0736-4b77-b91d-66b0bb9bfec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:52.210404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:52.214882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875f8fa-41fa-4570-90be-128314c896c7 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:52.228604 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 494/1007] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:52.229712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5bee1-0736-4b77-b91d-66b0bb9bfec4 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:52.254867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a13a7f-6264-49c9-9da8-ba0cd859965c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:52.255644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a13a7f-6264-49c9-9da8-ba0cd859965c None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.255961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a13a7f-6264-49c9-9da8-ba0cd859965c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.256236 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a13a7f-6264-49c9-9da8-ba0cd859965c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.256984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a13a7f-6264-49c9-9da8-ba0cd859965c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:52.258355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a13a7f-6264-49c9-9da8-ba0cd859965c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:52.274532 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 514/1008] 10.222.0.60 () {60 vars in 1144 bytes} [Thu Jul 21 20:39:52 2022] POST /identity/v3/endpoints => generated 325 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:39:52.275318 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a13a7f-6264-49c9-9da8-ba0cd859965c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:52.369241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5bee1-0736-4b77-b91d-66b0bb9bfec4 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:52.381686 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:52.426192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a13a7f-6264-49c9-9da8-ba0cd859965c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:52.433171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffdf95-331b-4af5-9ec5-8522a3c7243b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:52.433845 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffdf95-331b-4af5-9ec5-8522a3c7243b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.434131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffdf95-331b-4af5-9ec5-8522a3c7243b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.434407 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffdf95-331b-4af5-9ec5-8522a3c7243b None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.435125 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdffdf95-331b-4af5-9ec5-8522a3c7243b None admin] RBAC: Adding query filter params (type=compute) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:52.435430 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdffdf95-331b-4af5-9ec5-8522a3c7243b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:52.436860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdffdf95-331b-4af5-9ec5-8522a3c7243b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:52.451764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 495/1009] 10.222.0.60 () {56 vars in 1112 bytes} [Thu Jul 21 20:39:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:52.455058 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffdf95-331b-4af5-9ec5-8522a3c7243b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:52.482839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04b111-d8a3-4d3f-b681-de080a6786de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:52.483648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04b111-d8a3-4d3f-b681-de080a6786de None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.484048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04b111-d8a3-4d3f-b681-de080a6786de None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.484337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04b111-d8a3-4d3f-b681-de080a6786de None admin] PATH_INFO: `/v3/services/image` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.485001 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04b111-d8a3-4d3f-b681-de080a6786de None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:52.486391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04b111-d8a3-4d3f-b681-de080a6786de None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-1c04b111-d8a3-4d3f-b681-de080a6786de None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:52.499283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:52.500597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:52.501625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:52.501625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:52.501625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:52.501625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:52.501625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:52.501625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:52.501625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:52.501625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:52.501625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:52.501625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:52.501625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:52.501625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:52.501625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:39:52.501625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:52.501625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 515/1010] 10.222.0.60 () {56 vars in 1105 bytes} [Thu Jul 21 20:39:52 2022] GET /identity/v3/services/image => generated 86 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:52.507740 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04b111-d8a3-4d3f-b681-de080a6786de None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:52.525605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:52.586808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffdf95-331b-4af5-9ec5-8522a3c7243b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:52.625186 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae5ebf-8328-4a31-83f5-e2159903cf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:52.625431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04b111-d8a3-4d3f-b681-de080a6786de None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:52.625851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae5ebf-8328-4a31-83f5-e2159903cf75 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.626135 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae5ebf-8328-4a31-83f5-e2159903cf75 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.626411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae5ebf-8328-4a31-83f5-e2159903cf75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.627284 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae5ebf-8328-4a31-83f5-e2159903cf75 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:52.629587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae5ebf-8328-4a31-83f5-e2159903cf75 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:52.658901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 496/1011] 10.222.0.60 () {60 vars in 1144 bytes} [Thu Jul 21 20:39:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:52.680276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded5263-b8a7-4136-aac7-d594bd5e03e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:52.680974 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded5263-b8a7-4136-aac7-d594bd5e03e9 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.681261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded5263-b8a7-4136-aac7-d594bd5e03e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.681538 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded5263-b8a7-4136-aac7-d594bd5e03e9 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.682273 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded5263-b8a7-4136-aac7-d594bd5e03e9 None admin] RBAC: Adding query filter params (name=image) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:52.682577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded5263-b8a7-4136-aac7-d594bd5e03e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:52.684033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded5263-b8a7-4136-aac7-d594bd5e03e9 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:52.690290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 516/1012] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:52.697063 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae5ebf-8328-4a31-83f5-e2159903cf75 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:52.819867 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:52.824658 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae5ebf-8328-4a31-83f5-e2159903cf75 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:52.868393 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74884e45-2b32-48bb-8211-c1dda59a7d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:52.869328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74884e45-2b32-48bb-8211-c1dda59a7d9f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:52.869610 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74884e45-2b32-48bb-8211-c1dda59a7d9f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:52.869888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74884e45-2b32-48bb-8211-c1dda59a7d9f None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:52.870606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74884e45-2b32-48bb-8211-c1dda59a7d9f None admin] RBAC: Adding query filter params (type=image) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:52.870912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74884e45-2b32-48bb-8211-c1dda59a7d9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:52.872325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74884e45-2b32-48bb-8211-c1dda59a7d9f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:52.882709 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 497/1013] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:52 2022] GET /identity/v3/services?type=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:54.418941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8be65-3c90-4597-a0b1-a0a52b878dae None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.419480 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8be65-3c90-4597-a0b1-a0a52b878dae None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.419480 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8be65-3c90-4597-a0b1-a0a52b878dae None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.421065 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 517/1014] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:54.427119 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34c50c-1185-47f9-82b1-6d5a8d68f6ba None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.427547 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34c50c-1185-47f9-82b1-6d5a8d68f6ba None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.427755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34c50c-1185-47f9-82b1-6d5a8d68f6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.445220 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-0c34c50c-1185-47f9-82b1-6d5a8d68f6ba None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:54.446428 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-0c34c50c-1185-47f9-82b1-6d5a8d68f6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:54.531116 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 498/1015] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:54 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:54.555073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8be65-3c90-4597-a0b1-a0a52b878dae None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:54.564907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:54.680351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8be65-3c90-4597-a0b1-a0a52b878dae None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:54.720122 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a0bdd-d144-4abd-bdbe-3ff2772a7aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:54.720931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a0bdd-d144-4abd-bdbe-3ff2772a7aac None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.721213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a0bdd-d144-4abd-bdbe-3ff2772a7aac None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.721491 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a0bdd-d144-4abd-bdbe-3ff2772a7aac None admin] PATH_INFO: `/v3/domains/default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.723310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a0bdd-d144-4abd-bdbe-3ff2772a7aac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:54.724789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a0bdd-d144-4abd-bdbe-3ff2772a7aac None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:54.731374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 518/1016] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:54.732632 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34c50c-1185-47f9-82b1-6d5a8d68f6ba None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:54.838765 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:54.846788 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34c50c-1185-47f9-82b1-6d5a8d68f6ba None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:54.883605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b21718-3bc4-42bb-834b-6ff2c7235cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:54.884314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b21718-3bc4-42bb-834b-6ff2c7235cef None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.884598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b21718-3bc4-42bb-834b-6ff2c7235cef None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.884874 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b21718-3bc4-42bb-834b-6ff2c7235cef None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.885615 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b21718-3bc4-42bb-834b-6ff2c7235cef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:54.887069 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b21718-3bc4-42bb-834b-6ff2c7235cef None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:54.915791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 499/1017] 10.222.0.60 () {60 vars in 1141 bytes} [Thu Jul 21 20:39:54 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:39:54.945851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b213a-45db-48b4-a5d6-d9a90ad2fad0 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.946029 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b213a-45db-48b4-a5d6-d9a90ad2fad0 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.946315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b213a-45db-48b4-a5d6-d9a90ad2fad0 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.947970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 519/1018] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:54.953762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5363c-c093-4c87-8cf5-b1633df62aed None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.954057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5363c-c093-4c87-8cf5-b1633df62aed None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.954371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5363c-c093-4c87-8cf5-b1633df62aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.967947 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a125e0-114c-4aa3-961a-55cf4f0b2d6d None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.968177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a125e0-114c-4aa3-961a-55cf4f0b2d6d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.968465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a125e0-114c-4aa3-961a-55cf4f0b2d6d None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.969950 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 520/1019] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:39:54.974078 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-1eb5363c-c093-4c87-8cf5-b1633df62aed None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:54.975333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1eb5363c-c093-4c87-8cf5-b1633df62aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:54.976290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cfdfa-6c13-4799-abe5-c2cada0b6898 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:54.976583 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cfdfa-6c13-4799-abe5-c2cada0b6898 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:54.976863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cfdfa-6c13-4799-abe5-c2cada0b6898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:54.995570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d29cfdfa-6c13-4799-abe5-c2cada0b6898 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:54.996853 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d29cfdfa-6c13-4799-abe5-c2cada0b6898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:55.063742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 500/1020] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:54 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:55.081135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 521/1021] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:54 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:55.091733 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5363c-c093-4c87-8cf5-b1633df62aed None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:55.105238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cfdfa-6c13-4799-abe5-c2cada0b6898 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:55.108413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:55.184914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:55.221836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5363c-c093-4c87-8cf5-b1633df62aed None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:55.229632 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cfdfa-6c13-4799-abe5-c2cada0b6898 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:55.268412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdac9c8-7961-4167-b299-a13e3650b3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:55.269236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdac9c8-7961-4167-b299-a13e3650b3d6 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:55.269539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdac9c8-7961-4167-b299-a13e3650b3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:55.269705 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d8f96-7c47-4fbe-b18b-54e00619c529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:55.269815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdac9c8-7961-4167-b299-a13e3650b3d6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:55.270492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d8f96-7c47-4fbe-b18b-54e00619c529 None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:55.270745 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d8f96-7c47-4fbe-b18b-54e00619c529 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:55.271024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d8f96-7c47-4fbe-b18b-54e00619c529 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:55.271793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d8f96-7c47-4fbe-b18b-54e00619c529 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:55.273177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d8f96-7c47-4fbe-b18b-54e00619c529 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:55.284421 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdac9c8-7961-4167-b299-a13e3650b3d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:39:55.288008 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 522/1022] 10.222.0.60 () {60 vars in 1141 bytes} [Thu Jul 21 20:39:55 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-bbdac9c8-7961-4167-b299-a13e3650b3d6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:39:55.293868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:55.294875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:55.295757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:55.295757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:55.295757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:55.295757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:55.295757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:39:55.295757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:39:55.295757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:39:55.295757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:39:55.295757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:39:55.295757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:39:55.295757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:39:55.295757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:55.303604 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 501/1023] 10.222.0.60 () {56 vars in 1120 bytes} [Thu Jul 21 20:39:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:55.304770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d8f96-7c47-4fbe-b18b-54e00619c529 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:55.383955 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:55.443492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d8f96-7c47-4fbe-b18b-54e00619c529 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:55.489081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3d82d-ef74-46d8-8adc-f66b7f0b89bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:55.489929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3d82d-ef74-46d8-8adc-f66b7f0b89bf None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:55.490213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3d82d-ef74-46d8-8adc-f66b7f0b89bf None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:55.490498 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3d82d-ef74-46d8-8adc-f66b7f0b89bf None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:55.491265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3d82d-ef74-46d8-8adc-f66b7f0b89bf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:55.491572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3d82d-ef74-46d8-8adc-f66b7f0b89bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:55.493028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3d82d-ef74-46d8-8adc-f66b7f0b89bf None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:55.502197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 523/1024] 10.222.0.60 () {56 vars in 1115 bytes} [Thu Jul 21 20:39:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:55.511124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdac9c8-7961-4167-b299-a13e3650b3d6 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:55.641076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdac9c8-7961-4167-b299-a13e3650b3d6 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:55.681222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658c344-5a00-481b-992e-ddc96469da39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:55.682032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658c344-5a00-481b-992e-ddc96469da39 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:55.682315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658c344-5a00-481b-992e-ddc96469da39 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:55.682592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658c344-5a00-481b-992e-ddc96469da39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:55.688272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:55.698129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658c344-5a00-481b-992e-ddc96469da39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:55.699606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658c344-5a00-481b-992e-ddc96469da39 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:55.717793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-2658c344-5a00-481b-992e-ddc96469da39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:55.717793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:55.717793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:55.717793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:55.717793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:55.717793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:55.717793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:55.717793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:55.717793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:55.717793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:55.717793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:55.717793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:55.717793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:55.717793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:55.717793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:55.718791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:55.725190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:55.725190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:55.725190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:55.725190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:55.725190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 502/1025] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:55.727182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3d82d-ef74-46d8-8adc-f66b7f0b89bf None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:55.842294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3d82d-ef74-46d8-8adc-f66b7f0b89bf None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:55.871243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:55.889707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a0c98-d428-40c1-80a1-563ff28086fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:55.890375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a0c98-d428-40c1-80a1-563ff28086fd None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:55.890661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a0c98-d428-40c1-80a1-563ff28086fd None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:55.890936 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a0c98-d428-40c1-80a1-563ff28086fd None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:55.891663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a0c98-d428-40c1-80a1-563ff28086fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:55.892005 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a0c98-d428-40c1-80a1-563ff28086fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:55.893373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a0c98-d428-40c1-80a1-563ff28086fd None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:55.911561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 524/1026] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:55.912841 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658c344-5a00-481b-992e-ddc96469da39 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:56.022296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658c344-5a00-481b-992e-ddc96469da39 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:56.057657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393ab18-7267-4178-8f51-1e1e3281af16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:56.058463 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393ab18-7267-4178-8f51-1e1e3281af16 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:56.058746 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393ab18-7267-4178-8f51-1e1e3281af16 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:56.059019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393ab18-7267-4178-8f51-1e1e3281af16 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:56.068181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393ab18-7267-4178-8f51-1e1e3281af16 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:56.069652 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393ab18-7267-4178-8f51-1e1e3281af16 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-3393ab18-7267-4178-8f51-1e1e3281af16 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:39:56.086251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:56.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:56.092649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:56.092649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:56.092649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:39:56.092649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:39:56.092649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:39:56.092649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:39:56.092649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:39:56.092649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:39:56.092649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:39:56.092649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:39:56.092649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 503/1027] 10.222.0.60 () {56 vars in 1093 bytes} [Thu Jul 21 20:39:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:56.095182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a0c98-d428-40c1-80a1-563ff28086fd None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:56.210183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a0c98-d428-40c1-80a1-563ff28086fd None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:56.247327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd8b43-3379-49ee-9a0b-f429248bdc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:56.248030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd8b43-3379-49ee-9a0b-f429248bdc0b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:56.248314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd8b43-3379-49ee-9a0b-f429248bdc0b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:56.248591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd8b43-3379-49ee-9a0b-f429248bdc0b None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:56.249437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd8b43-3379-49ee-9a0b-f429248bdc0b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:56.249753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd8b43-3379-49ee-9a0b-f429248bdc0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:56.251187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd8b43-3379-49ee-9a0b-f429248bdc0b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:56.275250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 525/1028] 10.222.0.60 () {56 vars in 1133 bytes} [Thu Jul 21 20:39:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:56.276583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393ab18-7267-4178-8f51-1e1e3281af16 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:56.331529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:56.386758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393ab18-7267-4178-8f51-1e1e3281af16 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:56.429915 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef5701-4ddd-441e-9b59-6caa1aa85dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:56.430582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef5701-4ddd-441e-9b59-6caa1aa85dd5 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:56.430860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef5701-4ddd-441e-9b59-6caa1aa85dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:56.431133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef5701-4ddd-441e-9b59-6caa1aa85dd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:56.508471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef5701-4ddd-441e-9b59-6caa1aa85dd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:56.509927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef5701-4ddd-441e-9b59-6caa1aa85dd5 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:56.571255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:56.637144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 504/1029] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:39:56 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 20:39:56.638284 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd8b43-3379-49ee-9a0b-f429248bdc0b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:56.686552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:56.754807 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd8b43-3379-49ee-9a0b-f429248bdc0b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:56.792055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338b63a-9cf0-4bfd-8d41-7faaf7276f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:56.793047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338b63a-9cf0-4bfd-8d41-7faaf7276f43 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:56.793469 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338b63a-9cf0-4bfd-8d41-7faaf7276f43 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:56.793884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338b63a-9cf0-4bfd-8d41-7faaf7276f43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:56.805421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338b63a-9cf0-4bfd-8d41-7faaf7276f43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:56.806848 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338b63a-9cf0-4bfd-8d41-7faaf7276f43 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:56.825910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-6338b63a-9cf0-4bfd-8d41-7faaf7276f43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:56.825910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:56.825910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:56.825910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:56.825910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:56.825910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:56.825910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:56.825910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:56.825910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:56.825910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:56.825910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:56.825910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:56.825910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:39:56.825910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:39:56.825910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:56.826874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:56.832965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:39:56.832965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:39:56.832965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:39:56.832965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:56.832965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 526/1030] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:39:56.833506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef5701-4ddd-441e-9b59-6caa1aa85dd5 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:56.954620 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef5701-4ddd-441e-9b59-6caa1aa85dd5 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:56.999848 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637aed9-7518-40a9-800f-f9b100ea28ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:57.000532 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637aed9-7518-40a9-800f-f9b100ea28ba None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:57.000843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637aed9-7518-40a9-800f-f9b100ea28ba None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:57.001093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637aed9-7518-40a9-800f-f9b100ea28ba None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:57.001833 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637aed9-7518-40a9-800f-f9b100ea28ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:57.002142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637aed9-7518-40a9-800f-f9b100ea28ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:57.003524 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637aed9-7518-40a9-800f-f9b100ea28ba None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:57.023302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 505/1031] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:57.024691 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338b63a-9cf0-4bfd-8d41-7faaf7276f43 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:57.091523 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa5652-bf2c-405d-b88b-fd8702d93cf0 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:57.091868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa5652-bf2c-405d-b88b-fd8702d93cf0 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:57.092110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa5652-bf2c-405d-b88b-fd8702d93cf0 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:57.093647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 506/1032] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:57.098925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a84e7d-90d5-4261-a3d1-6fbd8c5d3d4c None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:57.099232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a84e7d-90d5-4261-a3d1-6fbd8c5d3d4c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:57.099513 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a84e7d-90d5-4261-a3d1-6fbd8c5d3d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:57.119091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-32a84e7d-90d5-4261-a3d1-6fbd8c5d3d4c None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:57.120311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-32a84e7d-90d5-4261-a3d1-6fbd8c5d3d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:57.122488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:57.137686 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:57.148758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338b63a-9cf0-4bfd-8d41-7faaf7276f43 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:57.187432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75880ef9-1f1c-4d32-93ed-2721a4bdde74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:57.188280 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75880ef9-1f1c-4d32-93ed-2721a4bdde74 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:57.188563 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75880ef9-1f1c-4d32-93ed-2721a4bdde74 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:57.188833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75880ef9-1f1c-4d32-93ed-2721a4bdde74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:57.196862 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75880ef9-1f1c-4d32-93ed-2721a4bdde74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:57.198314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75880ef9-1f1c-4d32-93ed-2721a4bdde74 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:57.212424 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 507/1033] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:57.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-75880ef9-1f1c-4d32-93ed-2721a4bdde74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:57.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:57.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:57.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:57.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:57.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:57.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:57.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:57.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:57.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:57.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:57.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:57.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:39:57.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:39:57.220961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:57.228328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:57.229597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:39:57.229597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:39:57.229597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:39:57.229597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:39:57.229597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:39:57.229597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:39:57.229597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:39:57.229597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:57.229597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 527/1034] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:39:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:57.230096 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a84e7d-90d5-4261-a3d1-6fbd8c5d3d4c None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:57.242402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75880ef9-1f1c-4d32-93ed-2721a4bdde74 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:57.345344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a84e7d-90d5-4261-a3d1-6fbd8c5d3d4c None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:57.370649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75880ef9-1f1c-4d32-93ed-2721a4bdde74 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:57.385135 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf37352-cda8-4f7b-b50d-ed99ef91b774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:57.386130 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf37352-cda8-4f7b-b50d-ed99ef91b774 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:57.386590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf37352-cda8-4f7b-b50d-ed99ef91b774 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:57.387033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf37352-cda8-4f7b-b50d-ed99ef91b774 None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:57.388049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf37352-cda8-4f7b-b50d-ed99ef91b774 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:57.388534 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf37352-cda8-4f7b-b50d-ed99ef91b774 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:57.390163 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf37352-cda8-4f7b-b50d-ed99ef91b774 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:57.400913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:57.413711 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 508/1035] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:39:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:57.413942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8f520-473b-4814-a7ea-c3a0deda069c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:57.414637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8f520-473b-4814-a7ea-c3a0deda069c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:57.414930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8f520-473b-4814-a7ea-c3a0deda069c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:57.415207 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8f520-473b-4814-a7ea-c3a0deda069c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:57.416500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8f520-473b-4814-a7ea-c3a0deda069c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:57.416965 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf37352-cda8-4f7b-b50d-ed99ef91b774 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:57.418376 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8f520-473b-4814-a7ea-c3a0deda069c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:57.424720 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 528/1036] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:39:57.427233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8f520-473b-4814-a7ea-c3a0deda069c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:57.451187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:57.529244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf37352-cda8-4f7b-b50d-ed99ef91b774 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:57.541524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8f520-473b-4814-a7ea-c3a0deda069c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:57.565602 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3425ae-032c-4e15-9692-084da3b325af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:57.566840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3425ae-032c-4e15-9692-084da3b325af None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:57.567228 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3425ae-032c-4e15-9692-084da3b325af None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:57.567655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3425ae-032c-4e15-9692-084da3b325af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:57.568531 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3425ae-032c-4e15-9692-084da3b325af None admin] RBAC: Adding query filter params (role.id=952e60f3db5044afa8b3043155a1ccca, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=77812b356c974d2b80102217956169db) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:57.568842 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3425ae-032c-4e15-9692-084da3b325af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:57.570268 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3425ae-032c-4e15-9692-084da3b325af None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:57.587738 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 509/1037] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:39:57 2022] GET /identity/v3/role_assignments?user.id=77812b356c974d2b80102217956169db&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=952e60f3db5044afa8b3043155a1ccca => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:57.588410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb91f4-819c-4d89-a310-9859ce484f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:57.588410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb91f4-819c-4d89-a310-9859ce484f5a None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:57.588410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb91f4-819c-4d89-a310-9859ce484f5a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:57.588410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb91f4-819c-4d89-a310-9859ce484f5a None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:57.588790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298d4d8-044e-4a0b-9112-8c961548cf52 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:57.588790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298d4d8-044e-4a0b-9112-8c961548cf52 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:57.588790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298d4d8-044e-4a0b-9112-8c961548cf52 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:57.589044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb91f4-819c-4d89-a310-9859ce484f5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:57.589114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 510/1038] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:57.589211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb91f4-819c-4d89-a310-9859ce484f5a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:57.593435 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30536e19-e961-4c5f-bd00-a518b4f901a8 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:57.593435 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30536e19-e961-4c5f-bd00-a518b4f901a8 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:57.593435 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30536e19-e961-4c5f-bd00-a518b4f901a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:57.605907 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-30536e19-e961-4c5f-bd00-a518b4f901a8 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:57.607251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-30536e19-e961-4c5f-bd00-a518b4f901a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:57.607985 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 529/1039] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:39:57 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:39:57.686727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:57.705725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 511/1040] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:57.729468 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb91f4-819c-4d89-a310-9859ce484f5a None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:57.823703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:57.854780 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb91f4-819c-4d89-a310-9859ce484f5a None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:57.893483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377d44f-c5aa-42ed-8cc1-feea070c0618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:57.894337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377d44f-c5aa-42ed-8cc1-feea070c0618 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:57.894624 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377d44f-c5aa-42ed-8cc1-feea070c0618 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:57.894897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377d44f-c5aa-42ed-8cc1-feea070c0618 None admin] PATH_INFO: `/v3/services/image` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:57.895563 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377d44f-c5aa-42ed-8cc1-feea070c0618 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:57.896984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377d44f-c5aa-42ed-8cc1-feea070c0618 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-5377d44f-c5aa-42ed-8cc1-feea070c0618 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:39:57.909282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:39:57.910894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:39:57.911882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:39:57.911882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:39:57.911882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:39:57.911882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:39:57.911882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:39:57.911882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:39:57.911882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:39:57.911882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:39:57.911882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:39:57.911882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:39:57.911882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:39:57.911882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:39:57.911882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:39:57.911882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:39:57.911882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 530/1041] 10.222.0.60 () {56 vars in 1105 bytes} [Thu Jul 21 20:39:57 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:39:57.917757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30536e19-e961-4c5f-bd00-a518b4f901a8 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:58.028253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30536e19-e961-4c5f-bd00-a518b4f901a8 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:58.069501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4d442-2cf2-46cc-bd65-c1299b7a8726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:58.070419 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4d442-2cf2-46cc-bd65-c1299b7a8726 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:58.070742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4d442-2cf2-46cc-bd65-c1299b7a8726 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:58.071119 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4d442-2cf2-46cc-bd65-c1299b7a8726 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:58.071926 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4d442-2cf2-46cc-bd65-c1299b7a8726 None admin] RBAC: Adding query filter params (name=image) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:58.072301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4d442-2cf2-46cc-bd65-c1299b7a8726 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:58.073856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4d442-2cf2-46cc-bd65-c1299b7a8726 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:58.081351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 512/1042] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:58.086078 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377d44f-c5aa-42ed-8cc1-feea070c0618 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:58.209994 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377d44f-c5aa-42ed-8cc1-feea070c0618 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:58.246668 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1b222-899b-4f3d-bb09-016dbd4d9a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:58.247339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1b222-899b-4f3d-bb09-016dbd4d9a5f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:58.247623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1b222-899b-4f3d-bb09-016dbd4d9a5f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:58.247919 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1b222-899b-4f3d-bb09-016dbd4d9a5f None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:58.248630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1b222-899b-4f3d-bb09-016dbd4d9a5f None admin] RBAC: Adding query filter params (type=image) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:58.248931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1b222-899b-4f3d-bb09-016dbd4d9a5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:58.250307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1b222-899b-4f3d-bb09-016dbd4d9a5f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:58.256634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 531/1043] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:39:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:58.262402 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4d442-2cf2-46cc-bd65-c1299b7a8726 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:58.325084 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:58.377651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4d442-2cf2-46cc-bd65-c1299b7a8726 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:58.426981 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894328b-97d9-4222-ae61-a65582afb48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:58.427644 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894328b-97d9-4222-ae61-a65582afb48d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:58.427998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894328b-97d9-4222-ae61-a65582afb48d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:58.428222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894328b-97d9-4222-ae61-a65582afb48d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:58.429188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894328b-97d9-4222-ae61-a65582afb48d None admin] RBAC: Adding query filter params (interface=public, service_id=5426699376694f22b6df6b222b8f35c1, region_id=RegionOne) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:39:58.429498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894328b-97d9-4222-ae61-a65582afb48d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:58.430870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894328b-97d9-4222-ae61-a65582afb48d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:58.442627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 513/1044] 10.222.0.60 () {56 vars in 1251 bytes} [Thu Jul 21 20:39:58 2022] GET /identity/v3/endpoints?service_id=5426699376694f22b6df6b222b8f35c1&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:39:58.442817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1b222-899b-4f3d-bb09-016dbd4d9a5f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:58.446596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:58.558132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1b222-899b-4f3d-bb09-016dbd4d9a5f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:39:58.602302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910b47c-3c93-47ba-a6b7-ec328f42c71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:39:58.602901 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910b47c-3c93-47ba-a6b7-ec328f42c71c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:58.603185 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910b47c-3c93-47ba-a6b7-ec328f42c71c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:58.603460 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910b47c-3c93-47ba-a6b7-ec328f42c71c None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:58.604441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910b47c-3c93-47ba-a6b7-ec328f42c71c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:39:58.606284 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910b47c-3c93-47ba-a6b7-ec328f42c71c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:39:58.619821 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 532/1045] 10.222.0.60 () {56 vars in 1087 bytes} [Thu Jul 21 20:39:58 2022] GET /identity/v3/services => generated 1957 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:39:59.716845 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5defa3-858d-407a-b6c7-0cb4ac5fc7ed None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:59.717336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5defa3-858d-407a-b6c7-0cb4ac5fc7ed None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:59.717336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5defa3-858d-407a-b6c7-0cb4ac5fc7ed None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:59.718865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 514/1046] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:59.724855 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2973f45-ea35-442a-9678-fe730a8185b2 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:59.725317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2973f45-ea35-442a-9678-fe730a8185b2 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:59.725415 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2973f45-ea35-442a-9678-fe730a8185b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:59.742942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-a2973f45-ea35-442a-9678-fe730a8185b2 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:59.744192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a2973f45-ea35-442a-9678-fe730a8185b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:59.813589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde7bf2-843c-4d5c-ad3a-fa8ec754d937 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:59.813872 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde7bf2-843c-4d5c-ad3a-fa8ec754d937 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:59.814145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde7bf2-843c-4d5c-ad3a-fa8ec754d937 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:59.815512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 515/1047] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:39:59.820869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c5b2d-aead-43f2-a45f-cb61121e8500 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:39:59.821155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c5b2d-aead-43f2-a45f-cb61121e8500 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:39:59.821432 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c5b2d-aead-43f2-a45f-cb61121e8500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:39:59.824907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:39:59.835455 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 533/1048] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:39:59.840052 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d62c5b2d-aead-43f2-a45f-cb61121e8500 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:39:59.841264 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d62c5b2d-aead-43f2-a45f-cb61121e8500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:39:59.862066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2973f45-ea35-442a-9678-fe730a8185b2 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:59.916955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 516/1049] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:39:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:39:59.943854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c5b2d-aead-43f2-a45f-cb61121e8500 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:39:59.986180 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2973f45-ea35-442a-9678-fe730a8185b2 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:00.002749 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:00.036695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c91d1e-8522-4bcf-a859-7f4b3d0d78f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:00.037525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c91d1e-8522-4bcf-a859-7f4b3d0d78f1 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.037810 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c91d1e-8522-4bcf-a859-7f4b3d0d78f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.038090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c91d1e-8522-4bcf-a859-7f4b3d0d78f1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.051163 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c91d1e-8522-4bcf-a859-7f4b3d0d78f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-57c91d1e-8522-4bcf-a859-7f4b3d0d78f1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:40:00.060026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:00.060982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:00.062071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:00.062071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:00.062071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:00.062071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:00.062071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:40:00.062071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:40:00.062071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:40:00.062071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:40:00.062071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:40:00.062071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:40:00.062071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:40:00.062071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:40:00.062071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 534/1050] 10.222.0.60 () {56 vars in 1120 bytes} [Thu Jul 21 20:39:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:40:00.067085 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c5b2d-aead-43f2-a45f-cb61121e8500 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:00.067729 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c91d1e-8522-4bcf-a859-7f4b3d0d78f1 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:00.107680 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee6abc-5c25-4f0d-b033-60463931d2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:00.108541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee6abc-5c25-4f0d-b033-60463931d2ee None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.108926 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee6abc-5c25-4f0d-b033-60463931d2ee None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.109170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee6abc-5c25-4f0d-b033-60463931d2ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.121613 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee6abc-5c25-4f0d-b033-60463931d2ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-fbee6abc-5c25-4f0d-b033-60463931d2ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:40:00.130077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:00.131064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:00.132001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:00.132001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:00.132001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:00.132001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:00.132001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:40:00.132001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:40:00.132001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:40:00.132001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:40:00.132001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:40:00.132001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:40:00.132001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:00.132001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:40:00.132001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 517/1051] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:39:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:40:00.135646 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:00.137721 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee6abc-5c25-4f0d-b033-60463931d2ee None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:00.189973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c91d1e-8522-4bcf-a859-7f4b3d0d78f1 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:00.228598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7ecdf-89b4-41d3-888e-d2e95995000d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:00.229653 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7ecdf-89b4-41d3-888e-d2e95995000d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.229935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7ecdf-89b4-41d3-888e-d2e95995000d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.230207 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7ecdf-89b4-41d3-888e-d2e95995000d None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.231013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7ecdf-89b4-41d3-888e-d2e95995000d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:00.231324 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7ecdf-89b4-41d3-888e-d2e95995000d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:00.232860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7ecdf-89b4-41d3-888e-d2e95995000d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:00.247503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 535/1052] 10.222.0.60 () {56 vars in 1115 bytes} [Thu Jul 21 20:40:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:00.249002 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7ecdf-89b4-41d3-888e-d2e95995000d None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:00.273171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee6abc-5c25-4f0d-b033-60463931d2ee None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:00.284401 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:00.310354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100f019-85b2-4e2b-8c90-c58df8a4c4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:00.311036 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100f019-85b2-4e2b-8c90-c58df8a4c4db None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.311321 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100f019-85b2-4e2b-8c90-c58df8a4c4db None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.311599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100f019-85b2-4e2b-8c90-c58df8a4c4db None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.312387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100f019-85b2-4e2b-8c90-c58df8a4c4db None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:00.312701 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100f019-85b2-4e2b-8c90-c58df8a4c4db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:00.314103 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100f019-85b2-4e2b-8c90-c58df8a4c4db None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:00.323555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 518/1053] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:40:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:00.329465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100f019-85b2-4e2b-8c90-c58df8a4c4db None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:00.379269 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7ecdf-89b4-41d3-888e-d2e95995000d None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:00.404288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:00.428045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d964896-46af-417b-8cae-e9cdb8dbf892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:00.428854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d964896-46af-417b-8cae-e9cdb8dbf892 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.429133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d964896-46af-417b-8cae-e9cdb8dbf892 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.429404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d964896-46af-417b-8cae-e9cdb8dbf892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.445028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d964896-46af-417b-8cae-e9cdb8dbf892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:00.446508 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d964896-46af-417b-8cae-e9cdb8dbf892 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:00.457515 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100f019-85b2-4e2b-8c90-c58df8a4c4db None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:00.466157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-5d964896-46af-417b-8cae-e9cdb8dbf892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:40:00.466157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:00.466157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:00.466157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:00.466157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:00.466157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:00.466157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:00.466157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:00.466157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:00.466157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:00.466157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:00.466157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:00.466157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:40:00.466157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:40:00.466157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:00.467112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:00.468037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:40:00.468037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:40:00.468037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:40:00.468037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:40:00.473375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 536/1054] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:40:00.474572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d964896-46af-417b-8cae-e9cdb8dbf892 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:00.499513 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fda3bf-e203-42b1-95ff-8bbf34749fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:00.500404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fda3bf-e203-42b1-95ff-8bbf34749fe3 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.500635 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fda3bf-e203-42b1-95ff-8bbf34749fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.501011 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fda3bf-e203-42b1-95ff-8bbf34749fe3 None admin] PATH_INFO: `/v3/users/0212eef31bc14b29812686ec3bc9c55b` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.517510 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fda3bf-e203-42b1-95ff-8bbf34749fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=0212eef31bc14b29812686ec3bc9c55b)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:00.518966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fda3bf-e203-42b1-95ff-8bbf34749fe3 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:00.524170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 519/1055] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:40:00 2022] GET /identity/v3/users/0212eef31bc14b29812686ec3bc9c55b => generated 285 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:00.527782 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fda3bf-e203-42b1-95ff-8bbf34749fe3 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:00.569600 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:00.600488 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d964896-46af-417b-8cae-e9cdb8dbf892 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:00.640256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28950bdb-a01b-4c70-b172-a8539aacaed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:00.641115 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28950bdb-a01b-4c70-b172-a8539aacaed9 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.641366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28950bdb-a01b-4c70-b172-a8539aacaed9 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.641639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28950bdb-a01b-4c70-b172-a8539aacaed9 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.642369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28950bdb-a01b-4c70-b172-a8539aacaed9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:00.642673 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28950bdb-a01b-4c70-b172-a8539aacaed9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:00.644095 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28950bdb-a01b-4c70-b172-a8539aacaed9 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:00.657344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 537/1056] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:00.663788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28950bdb-a01b-4c70-b172-a8539aacaed9 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:00.665474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fda3bf-e203-42b1-95ff-8bbf34749fe3 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:00.700088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:00.712897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe6307-a8fd-48c5-bf57-ce878aba8d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:00.712897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe6307-a8fd-48c5-bf57-ce878aba8d62 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.713344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe6307-a8fd-48c5-bf57-ce878aba8d62 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.713471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe6307-a8fd-48c5-bf57-ce878aba8d62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.781830 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28950bdb-a01b-4c70-b172-a8539aacaed9 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:00.803260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe6307-a8fd-48c5-bf57-ce878aba8d62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:00.804751 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe6307-a8fd-48c5-bf57-ce878aba8d62 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:00.818173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f719d3-951c-4c4c-9936-a56b0073c637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:00.818974 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f719d3-951c-4c4c-9936-a56b0073c637 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.819257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f719d3-951c-4c4c-9936-a56b0073c637 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.819533 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f719d3-951c-4c4c-9936-a56b0073c637 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.827051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f719d3-951c-4c4c-9936-a56b0073c637 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:00.828540 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f719d3-951c-4c4c-9936-a56b0073c637 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:00.835829 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-44f719d3-951c-4c4c-9936-a56b0073c637 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:40:00.843648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:00.844755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:00.845717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:00.845717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:00.845717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:40:00.845717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:40:00.845717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:40:00.845717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:40:00.845717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:40:00.845717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:40:00.845717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:40:00.845717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:40:00.845717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 538/1057] 10.222.0.60 () {56 vars in 1093 bytes} [Thu Jul 21 20:40:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:40:00.846340 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a3dc0-b089-44e2-9273-ad6dc17f6685 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.846408 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a3dc0-b089-44e2-9273-ad6dc17f6685 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.846773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a3dc0-b089-44e2-9273-ad6dc17f6685 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.849616 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 539/1058] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:00.852462 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a3dc0-b089-44e2-9273-ad6dc17f6685 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:00.919685 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 520/1059] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:40:00 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 20:40:00.921447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f6ba3-3ef0-4948-b465-3cd786682f19 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:00.921730 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f6ba3-3ef0-4948-b465-3cd786682f19 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:00.922013 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f6ba3-3ef0-4948-b465-3cd786682f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:00.940534 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-f78f6ba3-3ef0-4948-b465-3cd786682f19 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:00.941912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f78f6ba3-3ef0-4948-b465-3cd786682f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:00.964346 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:00.981127 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a3dc0-b089-44e2-9273-ad6dc17f6685 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.013086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5931bad-0363-4979-974b-753a9693138a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.013759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5931bad-0363-4979-974b-753a9693138a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.014046 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5931bad-0363-4979-974b-753a9693138a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.014327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5931bad-0363-4979-974b-753a9693138a None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.015170 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5931bad-0363-4979-974b-753a9693138a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:01.015481 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5931bad-0363-4979-974b-753a9693138a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.016956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5931bad-0363-4979-974b-753a9693138a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.022065 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 521/1060] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:01.022916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f6ba3-3ef0-4948-b465-3cd786682f19 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:01.036106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 540/1061] 10.222.0.60 () {56 vars in 1133 bytes} [Thu Jul 21 20:40:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:01.045785 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5931bad-0363-4979-974b-753a9693138a None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:01.098918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:01.140448 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f6ba3-3ef0-4948-b465-3cd786682f19 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.153809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5931bad-0363-4979-974b-753a9693138a None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.176233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55ce7d-aa32-48f6-b7ad-2f58aa5d2e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.177116 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55ce7d-aa32-48f6-b7ad-2f58aa5d2e0f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.177399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55ce7d-aa32-48f6-b7ad-2f58aa5d2e0f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.177677 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55ce7d-aa32-48f6-b7ad-2f58aa5d2e0f None admin] PATH_INFO: `/v3/projects/44d9127ed1f8417891c3f2106c514520` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.178924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55ce7d-aa32-48f6-b7ad-2f58aa5d2e0f None admin] RBAC: Authorizing `identity:get_project(project_id=44d9127ed1f8417891c3f2106c514520)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.180411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55ce7d-aa32-48f6-b7ad-2f58aa5d2e0f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.182179 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 522/1062] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:40:01 2022] GET /identity/v3/projects/44d9127ed1f8417891c3f2106c514520 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:01.182916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55ce7d-aa32-48f6-b7ad-2f58aa5d2e0f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:01.193800 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244db6c-a900-4f1f-a896-169d3368f445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.194594 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244db6c-a900-4f1f-a896-169d3368f445 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.194877 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244db6c-a900-4f1f-a896-169d3368f445 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.195156 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244db6c-a900-4f1f-a896-169d3368f445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.211268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244db6c-a900-4f1f-a896-169d3368f445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.212757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244db6c-a900-4f1f-a896-169d3368f445 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.234337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-d244db6c-a900-4f1f-a896-169d3368f445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:40:01.234337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:01.234337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:01.234337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:01.234337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:01.234337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:01.234337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:01.234337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:01.234337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:01.234337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:01.234337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:01.234337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:01.234337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:40:01.234337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:40:01.234337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:01.235335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:01.236513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:40:01.236513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:40:01.236513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:40:01.236513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:40:01.236513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 541/1063] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:40:01.236839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244db6c-a900-4f1f-a896-169d3368f445 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:01.239900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:01.300501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55ce7d-aa32-48f6-b7ad-2f58aa5d2e0f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.328148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:01.337402 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417701c-9143-4f0f-91a0-158a93f36d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.338232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417701c-9143-4f0f-91a0-158a93f36d7c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.338510 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417701c-9143-4f0f-91a0-158a93f36d7c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.338779 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417701c-9143-4f0f-91a0-158a93f36d7c None admin] PATH_INFO: `/v3/services/image` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.339427 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417701c-9143-4f0f-91a0-158a93f36d7c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.340832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417701c-9143-4f0f-91a0-158a93f36d7c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.352973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244db6c-a900-4f1f-a896-169d3368f445 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-5417701c-9143-4f0f-91a0-158a93f36d7c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:01.355181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:01.356190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:01.357086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:01.357086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:01.357086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:01.357086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:01.357086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:01.357086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:01.357086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:01.357086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:01.357086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:01.357086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:01.357086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:40:01.357086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:40:01.357086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 20:40:01.357086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:40:01.357086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 523/1064] 10.222.0.60 () {56 vars in 1105 bytes} [Thu Jul 21 20:40:01 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:40:01.357872 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417701c-9143-4f0f-91a0-158a93f36d7c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:01.390479 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d79ca-8a39-4b86-8f5c-ae3ade95173b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.391142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d79ca-8a39-4b86-8f5c-ae3ade95173b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.391450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d79ca-8a39-4b86-8f5c-ae3ade95173b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.391909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d79ca-8a39-4b86-8f5c-ae3ade95173b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.392602 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d79ca-8a39-4b86-8f5c-ae3ade95173b None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=44d9127ed1f8417891c3f2106c514520, user.id=0212eef31bc14b29812686ec3bc9c55b) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:01.392915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d79ca-8a39-4b86-8f5c-ae3ade95173b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.394492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d79ca-8a39-4b86-8f5c-ae3ade95173b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.408839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 542/1065] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:40:01 2022] GET /identity/v3/role_assignments?user.id=0212eef31bc14b29812686ec3bc9c55b&scope.project.id=44d9127ed1f8417891c3f2106c514520&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:01.408839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d79ca-8a39-4b86-8f5c-ae3ade95173b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:01.485508 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:01.495224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417701c-9143-4f0f-91a0-158a93f36d7c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.527391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d79ca-8a39-4b86-8f5c-ae3ade95173b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.539348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a2dcf-b214-49f3-a311-5bcf8d612b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.540499 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a2dcf-b214-49f3-a311-5bcf8d612b80 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.540977 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a2dcf-b214-49f3-a311-5bcf8d612b80 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.541440 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a2dcf-b214-49f3-a311-5bcf8d612b80 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.542483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a2dcf-b214-49f3-a311-5bcf8d612b80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:01.542976 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a2dcf-b214-49f3-a311-5bcf8d612b80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.544721 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a2dcf-b214-49f3-a311-5bcf8d612b80 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.561653 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 524/1066] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:01.567353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a2dcf-b214-49f3-a311-5bcf8d612b80 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:01.568838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130d2b2-2943-4cd7-8ea7-6d5873d165d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.570267 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130d2b2-2943-4cd7-8ea7-6d5873d165d4 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.570722 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130d2b2-2943-4cd7-8ea7-6d5873d165d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.571129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130d2b2-2943-4cd7-8ea7-6d5873d165d4 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.572231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130d2b2-2943-4cd7-8ea7-6d5873d165d4 None admin] RBAC: Adding query filter params (name=image) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:01.572551 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130d2b2-2943-4cd7-8ea7-6d5873d165d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.575047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130d2b2-2943-4cd7-8ea7-6d5873d165d4 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.579773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 543/1067] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:40:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:01.588272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130d2b2-2943-4cd7-8ea7-6d5873d165d4 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:01.645584 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:01.709710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a2dcf-b214-49f3-a311-5bcf8d612b80 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.714168 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130d2b2-2943-4cd7-8ea7-6d5873d165d4 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.744998 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:01.746879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5684fe8-2ec7-4fa3-b02b-9a2253d68a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.747840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5684fe8-2ec7-4fa3-b02b-9a2253d68a0b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.748134 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5684fe8-2ec7-4fa3-b02b-9a2253d68a0b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.748414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5684fe8-2ec7-4fa3-b02b-9a2253d68a0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.753037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c38ca-2119-4a45-8ebe-befed9e99b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.754500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c38ca-2119-4a45-8ebe-befed9e99b49 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.754952 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c38ca-2119-4a45-8ebe-befed9e99b49 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.755377 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c38ca-2119-4a45-8ebe-befed9e99b49 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.756575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c38ca-2119-4a45-8ebe-befed9e99b49 None admin] RBAC: Adding query filter params (type=image) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:01.756810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5684fe8-2ec7-4fa3-b02b-9a2253d68a0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.757217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c38ca-2119-4a45-8ebe-befed9e99b49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.758270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5684fe8-2ec7-4fa3-b02b-9a2253d68a0b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.759095 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c38ca-2119-4a45-8ebe-befed9e99b49 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.770487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 544/1068] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:40:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:01.771852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c38ca-2119-4a45-8ebe-befed9e99b49 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:01.773637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-f5684fe8-2ec7-4fa3-b02b-9a2253d68a0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:40:01.773637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:01.773637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:01.773637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:01.773637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:01.773637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:01.773637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:01.773637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:01.773637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:01.773637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:01.773637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:01.773637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:01.773637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:40:01.773637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:40:01.773637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:01.774512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:01.775406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:40:01.775406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:40:01.775406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:40:01.775406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:40:01.775406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:40:01.775406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:40:01.775406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:40:01.775406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:40:01.775406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 525/1069] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:40:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:40:01.780497 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5684fe8-2ec7-4fa3-b02b-9a2253d68a0b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:01.892390 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c38ca-2119-4a45-8ebe-befed9e99b49 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.904270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5684fe8-2ec7-4fa3-b02b-9a2253d68a0b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:01.919293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:01.939126 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10765209-98c9-4476-b898-66e31baf8404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.940020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10765209-98c9-4476-b898-66e31baf8404 None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.940374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10765209-98c9-4476-b898-66e31baf8404 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.940724 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10765209-98c9-4476-b898-66e31baf8404 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.943355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10765209-98c9-4476-b898-66e31baf8404 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.944850 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10765209-98c9-4476-b898-66e31baf8404 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.947064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5ab00-14e2-4574-9f88-3e91e19d1187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:01.947760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5ab00-14e2-4574-9f88-3e91e19d1187 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:01.948050 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5ab00-14e2-4574-9f88-3e91e19d1187 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:01.948333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5ab00-14e2-4574-9f88-3e91e19d1187 None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:01.949132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5ab00-14e2-4574-9f88-3e91e19d1187 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:01.949442 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5ab00-14e2-4574-9f88-3e91e19d1187 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:01.950874 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5ab00-14e2-4574-9f88-3e91e19d1187 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:01.964672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 526/1070] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:40:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:01.965626 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 545/1071] 10.222.0.60 () {60 vars in 1144 bytes} [Thu Jul 21 20:40:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:01.970658 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5ab00-14e2-4574-9f88-3e91e19d1187 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:02.110057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5ab00-14e2-4574-9f88-3e91e19d1187 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:02.162682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614db8f0-ed9f-4c51-a951-14724e7687f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:02.163507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614db8f0-ed9f-4c51-a951-14724e7687f6 None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:02.163834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614db8f0-ed9f-4c51-a951-14724e7687f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:02.164114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614db8f0-ed9f-4c51-a951-14724e7687f6 None admin] PATH_INFO: `/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/77812b356c974d2b80102217956169db/roles/952e60f3db5044afa8b3043155a1ccca` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:02.166446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614db8f0-ed9f-4c51-a951-14724e7687f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc61fe15894891a9054911321f4ac3, user_id=77812b356c974d2b80102217956169db, role_id=952e60f3db5044afa8b3043155a1ccca)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:02.167968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614db8f0-ed9f-4c51-a951-14724e7687f6 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:02.182823 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 527/1072] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:40:01 2022] PUT /identity/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/77812b356c974d2b80102217956169db/roles/952e60f3db5044afa8b3043155a1ccca => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:03.497625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f074f5-944d-4370-b420-5894c4e25f94 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:03.498176 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f074f5-944d-4370-b420-5894c4e25f94 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:03.498176 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f074f5-944d-4370-b420-5894c4e25f94 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:03.499693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 546/1073] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:03.505340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de7973-7375-4bee-8075-9ed84cb4387c None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:03.505803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de7973-7375-4bee-8075-9ed84cb4387c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:03.505898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de7973-7375-4bee-8075-9ed84cb4387c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:03.523315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-47de7973-7375-4bee-8075-9ed84cb4387c None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:03.524528 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-47de7973-7375-4bee-8075-9ed84cb4387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:03.549018 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:03.594944 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 528/1074] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:03.618122 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f074f5-944d-4370-b420-5894c4e25f94 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:03.672261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:03.734512 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f074f5-944d-4370-b420-5894c4e25f94 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:03.771391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2eebfb-2c1a-4e8d-921b-98149af2bcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:03.772252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2eebfb-2c1a-4e8d-921b-98149af2bcbf None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:03.772524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2eebfb-2c1a-4e8d-921b-98149af2bcbf None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:03.772803 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2eebfb-2c1a-4e8d-921b-98149af2bcbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:03.784268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2eebfb-2c1a-4e8d-921b-98149af2bcbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-4e2eebfb-2c1a-4e8d-921b-98149af2bcbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:40:03.792310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:03.793312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:03.794418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:03.794418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:03.794418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:03.794418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:03.794418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:40:03.794418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:40:03.794418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:40:03.794418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:40:03.794418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:40:03.794418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:40:03.794418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:03.794418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:40:03.794418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 547/1075] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:40:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:40:03.799589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de7973-7375-4bee-8075-9ed84cb4387c None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:03.921688 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de7973-7375-4bee-8075-9ed84cb4387c None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:03.971009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea75efc-b729-441e-b8ed-1487d2461eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:03.971612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea75efc-b729-441e-b8ed-1487d2461eba None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:03.971925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea75efc-b729-441e-b8ed-1487d2461eba None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:03.972771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea75efc-b729-441e-b8ed-1487d2461eba None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:03.973624 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea75efc-b729-441e-b8ed-1487d2461eba None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:03.974123 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea75efc-b729-441e-b8ed-1487d2461eba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:03.975522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea75efc-b729-441e-b8ed-1487d2461eba None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:03.990559 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 529/1076] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:40:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:03.992615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2eebfb-2c1a-4e8d-921b-98149af2bcbf None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.101265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2eebfb-2c1a-4e8d-921b-98149af2bcbf None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.115644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:04.139226 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54aa5d-11f9-45b5-89c1-b635f772cdb0 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.139611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54aa5d-11f9-45b5-89c1-b635f772cdb0 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.139874 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54aa5d-11f9-45b5-89c1-b635f772cdb0 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.141478 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 530/1077] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:04.142100 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceec00f-5c2d-44d8-ab3f-ec74d93a95c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.143044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceec00f-5c2d-44d8-ab3f-ec74d93a95c5 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.143330 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceec00f-5c2d-44d8-ab3f-ec74d93a95c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.143604 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceec00f-5c2d-44d8-ab3f-ec74d93a95c5 None admin] PATH_INFO: `/v3/users/0212eef31bc14b29812686ec3bc9c55b` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.145628 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceec00f-5c2d-44d8-ab3f-ec74d93a95c5 None admin] RBAC: Authorizing `identity:get_user(user_id=0212eef31bc14b29812686ec3bc9c55b)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.147224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceec00f-5c2d-44d8-ab3f-ec74d93a95c5 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.147542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4faf1-0b1b-4dcb-a660-45b2832f04cc None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.147869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4faf1-0b1b-4dcb-a660-45b2832f04cc None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.148152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4faf1-0b1b-4dcb-a660-45b2832f04cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.153748 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 548/1078] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:40:03 2022] GET /identity/v3/users/0212eef31bc14b29812686ec3bc9c55b => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:04.154566 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceec00f-5c2d-44d8-ab3f-ec74d93a95c5 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.169868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-63d4faf1-0b1b-4dcb-a660-45b2832f04cc None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:04.171080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-63d4faf1-0b1b-4dcb-a660-45b2832f04cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:04.187533 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:04.254701 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 531/1079] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:40:04.277623 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4faf1-0b1b-4dcb-a660-45b2832f04cc None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.277997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceec00f-5c2d-44d8-ab3f-ec74d93a95c5 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.316853 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1065e9-e684-403f-b12b-f5d3aa4d6147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.317769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1065e9-e684-403f-b12b-f5d3aa4d6147 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.318053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1065e9-e684-403f-b12b-f5d3aa4d6147 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.318332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1065e9-e684-403f-b12b-f5d3aa4d6147 None admin] PATH_INFO: `/v3/projects/44d9127ed1f8417891c3f2106c514520` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.319606 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1065e9-e684-403f-b12b-f5d3aa4d6147 None admin] RBAC: Authorizing `identity:get_project(project_id=44d9127ed1f8417891c3f2106c514520)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.321092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1065e9-e684-403f-b12b-f5d3aa4d6147 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.322834 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 549/1080] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:40:04 2022] GET /identity/v3/projects/44d9127ed1f8417891c3f2106c514520 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:04.324155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b029dd-51f3-4fb7-9d2f-94459b84898c None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.324447 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b029dd-51f3-4fb7-9d2f-94459b84898c None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.324725 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b029dd-51f3-4fb7-9d2f-94459b84898c None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.326127 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 550/1081] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:04.328742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b029dd-51f3-4fb7-9d2f-94459b84898c None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.384356 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:04.391492 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4faf1-0b1b-4dcb-a660-45b2832f04cc None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.433293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd8b79-42b3-4988-80f5-2fe38ec867e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.433963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd8b79-42b3-4988-80f5-2fe38ec867e2 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.434248 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd8b79-42b3-4988-80f5-2fe38ec867e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.434528 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd8b79-42b3-4988-80f5-2fe38ec867e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.448204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b029dd-51f3-4fb7-9d2f-94459b84898c None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.455276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:04.483541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64506de0-713b-4759-873e-79362a005825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.484471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64506de0-713b-4759-873e-79362a005825 None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.484759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64506de0-713b-4759-873e-79362a005825 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.485051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64506de0-713b-4759-873e-79362a005825 None admin] PATH_INFO: `/v3/projects/44d9127ed1f8417891c3f2106c514520/users/0212eef31bc14b29812686ec3bc9c55b/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.487051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64506de0-713b-4759-873e-79362a005825 None admin] RBAC: Authorizing `identity:create_grant(project_id=44d9127ed1f8417891c3f2106c514520, user_id=0212eef31bc14b29812686ec3bc9c55b, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.488560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64506de0-713b-4759-873e-79362a005825 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.502160 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 551/1082] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:40:04 2022] PUT /identity/v3/projects/44d9127ed1f8417891c3f2106c514520/users/0212eef31bc14b29812686ec3bc9c55b/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:40:04.503625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6c18f-546a-427e-97c6-9a6e4ef91362 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.503940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6c18f-546a-427e-97c6-9a6e4ef91362 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.504218 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6c18f-546a-427e-97c6-9a6e4ef91362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.523880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd8b79-42b3-4988-80f5-2fe38ec867e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.525289 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd8b79-42b3-4988-80f5-2fe38ec867e2 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.532277 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-19b6c18f-546a-427e-97c6-9a6e4ef91362 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:04.533517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-19b6c18f-546a-427e-97c6-9a6e4ef91362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:04.626143 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 552/1083] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:04.645923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 532/1084] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:40:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 20:40:04.651856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd8b79-42b3-4988-80f5-2fe38ec867e2 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.652039 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6c18f-546a-427e-97c6-9a6e4ef91362 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.697187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:04.718812 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:04.774839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6c18f-546a-427e-97c6-9a6e4ef91362 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.777298 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd8b79-42b3-4988-80f5-2fe38ec867e2 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.810640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2925aa-2a4d-48c6-b2b4-2daed534cea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.811905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2925aa-2a4d-48c6-b2b4-2daed534cea8 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.812378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2925aa-2a4d-48c6-b2b4-2daed534cea8 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.812937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2925aa-2a4d-48c6-b2b4-2daed534cea8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.813221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d90bf-5525-4567-ab25-1aa826a3bc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:04.814041 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d90bf-5525-4567-ab25-1aa826a3bc23 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:04.814314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d90bf-5525-4567-ab25-1aa826a3bc23 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:04.814895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d90bf-5525-4567-ab25-1aa826a3bc23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:04.826540 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2925aa-2a4d-48c6-b2b4-2daed534cea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:40:04.826731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d90bf-5525-4567-ab25-1aa826a3bc23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:04.828083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d90bf-5525-4567-ab25-1aa826a3bc23 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-4b2925aa-2a4d-48c6-b2b4-2daed534cea8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:40:04.836384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:04.837367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:04.838306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:04.838306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:04.838306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:04.838306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:04.838306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:40:04.838306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:40:04.838306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:40:04.838306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:40:04.838306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:40:04.838306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:40:04.838306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 20:40:04.838306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:40:04.846817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 553/1085] 10.222.0.60 () {56 vars in 1120 bytes} [Thu Jul 21 20:40:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:40:04.848521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2925aa-2a4d-48c6-b2b4-2daed534cea8 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.848799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-ac3d90bf-5525-4567-ab25-1aa826a3bc23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:40:04.848799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:04.848799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:04.848799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:04.848799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:04.848799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:04.848799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:04.848799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:04.848799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:04.848799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:04.848799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:04.848799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:04.848799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:40:04.848799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:40:04.848799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:04.849831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:04.850851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:40:04.850851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:40:04.850851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:40:04.850851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:40:04.850851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 533/1086] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:40:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:40:04.856196 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d90bf-5525-4567-ab25-1aa826a3bc23 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:04.970220 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:04.980710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d90bf-5525-4567-ab25-1aa826a3bc23 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:04.982548 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:04.982667 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2925aa-2a4d-48c6-b2b4-2daed534cea8 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:05.020483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d9c95-bab7-43ff-8fe5-f734da195628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:05.021419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c1840-7e2c-4dae-b2fe-c3856bb0f404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:05.021611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d9c95-bab7-43ff-8fe5-f734da195628 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:05.022030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d9c95-bab7-43ff-8fe5-f734da195628 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:05.022143 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c1840-7e2c-4dae-b2fe-c3856bb0f404 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:05.022366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c1840-7e2c-4dae-b2fe-c3856bb0f404 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:05.022647 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c1840-7e2c-4dae-b2fe-c3856bb0f404 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:05.022839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d9c95-bab7-43ff-8fe5-f734da195628 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:05.023443 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c1840-7e2c-4dae-b2fe-c3856bb0f404 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:05.023776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c1840-7e2c-4dae-b2fe-c3856bb0f404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:05.024062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d9c95-bab7-43ff-8fe5-f734da195628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:05.024506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d9c95-bab7-43ff-8fe5-f734da195628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:05.025228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c1840-7e2c-4dae-b2fe-c3856bb0f404 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:05.026231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d9c95-bab7-43ff-8fe5-f734da195628 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:05.036995 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 554/1087] 10.222.0.60 () {56 vars in 1115 bytes} [Thu Jul 21 20:40:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:05.043079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c1840-7e2c-4dae-b2fe-c3856bb0f404 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:05.044056 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 534/1088] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:40:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:05.173901 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c1840-7e2c-4dae-b2fe-c3856bb0f404 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:05.211847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503f6e7-ebe7-420b-a279-90e2de8b7c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:05.212649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503f6e7-ebe7-420b-a279-90e2de8b7c8b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:05.212930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503f6e7-ebe7-420b-a279-90e2de8b7c8b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:05.213201 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503f6e7-ebe7-420b-a279-90e2de8b7c8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:05.226520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503f6e7-ebe7-420b-a279-90e2de8b7c8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:05.228000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503f6e7-ebe7-420b-a279-90e2de8b7c8b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:05.246482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-5503f6e7-ebe7-420b-a279-90e2de8b7c8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:40:05.246482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:05.246482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:05.246482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:05.246482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:05.246482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:05.246482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:05.246482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:05.246482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:05.246482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:05.246482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:05.246482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:05.246482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:40:05.246482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:40:05.246482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:05.247431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:05.248387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:40:05.248387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:40:05.248387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:40:05.248387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:40:05.250957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 555/1089] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:40:05.254752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d9c95-bab7-43ff-8fe5-f734da195628 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:05.373263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d9c95-bab7-43ff-8fe5-f734da195628 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:05.410375 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5c94a-6822-401f-a4ab-4ccf70684a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:05.411039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5c94a-6822-401f-a4ab-4ccf70684a8e None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:05.411322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5c94a-6822-401f-a4ab-4ccf70684a8e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:05.411596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5c94a-6822-401f-a4ab-4ccf70684a8e None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:05.412539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5c94a-6822-401f-a4ab-4ccf70684a8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:05.412847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5c94a-6822-401f-a4ab-4ccf70684a8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:05.414222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5c94a-6822-401f-a4ab-4ccf70684a8e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:05.427060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 535/1090] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:05.432762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503f6e7-ebe7-420b-a279-90e2de8b7c8b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:05.446680 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:05.541541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503f6e7-ebe7-420b-a279-90e2de8b7c8b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:05.577253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12cebf-8fbf-4451-a73b-8c3578a85335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:05.578043 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12cebf-8fbf-4451-a73b-8c3578a85335 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:05.578321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12cebf-8fbf-4451-a73b-8c3578a85335 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:05.578597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12cebf-8fbf-4451-a73b-8c3578a85335 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:05.587211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12cebf-8fbf-4451-a73b-8c3578a85335 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:05.588702 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12cebf-8fbf-4451-a73b-8c3578a85335 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-2d12cebf-8fbf-4451-a73b-8c3578a85335 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:40:05.605243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:05.606331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:05.607329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:05.607329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:05.607329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:40:05.607329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:40:05.607329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:40:05.607329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:40:05.607329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:40:05.607329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:40:05.607329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 20:40:05.607329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:40:05.607329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 556/1091] 10.222.0.60 () {56 vars in 1093 bytes} [Thu Jul 21 20:40:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:40:05.612379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5c94a-6822-401f-a4ab-4ccf70684a8e None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:05.638820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:05.718941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5c94a-6822-401f-a4ab-4ccf70684a8e None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:05.755817 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb8059-9d22-4090-b62f-c77d50f4938b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:05.756685 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb8059-9d22-4090-b62f-c77d50f4938b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:05.756961 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb8059-9d22-4090-b62f-c77d50f4938b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:05.757232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb8059-9d22-4090-b62f-c77d50f4938b None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:05.758073 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb8059-9d22-4090-b62f-c77d50f4938b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:05.758378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb8059-9d22-4090-b62f-c77d50f4938b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:05.759839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb8059-9d22-4090-b62f-c77d50f4938b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:05.779699 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 536/1092] 10.222.0.60 () {56 vars in 1133 bytes} [Thu Jul 21 20:40:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:05.787352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12cebf-8fbf-4451-a73b-8c3578a85335 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:05.861126 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:05.893840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12cebf-8fbf-4451-a73b-8c3578a85335 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:05.928843 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c536fad-9691-4764-b65f-78c51695a05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:05.929492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c536fad-9691-4764-b65f-78c51695a05f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:05.929775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c536fad-9691-4764-b65f-78c51695a05f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:05.930050 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c536fad-9691-4764-b65f-78c51695a05f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:05.998725 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c536fad-9691-4764-b65f-78c51695a05f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:06.000211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c536fad-9691-4764-b65f-78c51695a05f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:06.068675 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:06.102901 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 557/1093] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:40:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 20:40:06.109020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb8059-9d22-4090-b62f-c77d50f4938b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:06.192737 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:06.212671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb8059-9d22-4090-b62f-c77d50f4938b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:06.248815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34f4e3-95ab-4b41-b4e7-8a51441458c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:06.249610 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34f4e3-95ab-4b41-b4e7-8a51441458c3 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:06.249892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34f4e3-95ab-4b41-b4e7-8a51441458c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:06.250167 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34f4e3-95ab-4b41-b4e7-8a51441458c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:06.261295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34f4e3-95ab-4b41-b4e7-8a51441458c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:06.262748 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34f4e3-95ab-4b41-b4e7-8a51441458c3 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:06.280786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-ce34f4e3-95ab-4b41-b4e7-8a51441458c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:40:06.280786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:06.280786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:06.280786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:06.280786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:06.280786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:06.280786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:06.280786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:06.280786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:06.280786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:06.280786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:06.280786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:06.280786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:40:06.280786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:40:06.280786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:06.281873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:06.282750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:40:06.282750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:40:06.282750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:40:06.282750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:40:06.282750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 537/1094] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:40:06.288380 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c536fad-9691-4764-b65f-78c51695a05f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:06.389466 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c536fad-9691-4764-b65f-78c51695a05f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:06.428588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887840da-c8b9-43c8-9808-7b421b78dd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:06.429252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887840da-c8b9-43c8-9808-7b421b78dd3c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:06.429533 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887840da-c8b9-43c8-9808-7b421b78dd3c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:06.429840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887840da-c8b9-43c8-9808-7b421b78dd3c None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:06.430850 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887840da-c8b9-43c8-9808-7b421b78dd3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:06.431155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887840da-c8b9-43c8-9808-7b421b78dd3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:06.432554 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887840da-c8b9-43c8-9808-7b421b78dd3c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:06.445086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 558/1095] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:06.450474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34f4e3-95ab-4b41-b4e7-8a51441458c3 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:06.553726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34f4e3-95ab-4b41-b4e7-8a51441458c3 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:06.583633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:06.588839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76693a-2790-491e-842f-952d90b47421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:06.589623 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76693a-2790-491e-842f-952d90b47421 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:06.589896 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76693a-2790-491e-842f-952d90b47421 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:06.590162 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76693a-2790-491e-842f-952d90b47421 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:06.597209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76693a-2790-491e-842f-952d90b47421 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:06.598651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76693a-2790-491e-842f-952d90b47421 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:06.613333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-5c76693a-2790-491e-842f-952d90b47421 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:40:06.613333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:06.613333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:06.613333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:06.613333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:06.613333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:06.613333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:06.613333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:06.613333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:06.613333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:06.613333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:06.613333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:06.613333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:40:06.613333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:40:06.613333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:06.614725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:06.615609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:40:06.615609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:40:06.615609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:40:06.615609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:40:06.615609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:40:06.615609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:40:06.615609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:40:06.615609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:40:06.615609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 538/1096] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:40:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:40:06.620488 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887840da-c8b9-43c8-9808-7b421b78dd3c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:06.638706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167a67e-8dc4-4f84-a5dc-7c4c69023fce None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:06.638996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167a67e-8dc4-4f84-a5dc-7c4c69023fce None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:06.639267 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167a67e-8dc4-4f84-a5dc-7c4c69023fce None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:06.640700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 539/1097] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:06.646394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d9cb0-2d06-4d51-bc91-114c78f2d132 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:06.646680 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d9cb0-2d06-4d51-bc91-114c78f2d132 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:06.646956 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d9cb0-2d06-4d51-bc91-114c78f2d132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:06.655543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:06.667401 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-707d9cb0-2d06-4d51-bc91-114c78f2d132 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:06.668652 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-707d9cb0-2d06-4d51-bc91-114c78f2d132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:06.733300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887840da-c8b9-43c8-9808-7b421b78dd3c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:06.743822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 540/1098] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:40:06.767890 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d9cb0-2d06-4d51-bc91-114c78f2d132 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:06.771316 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbba578-10f1-4859-855a-a67cea9bbf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:06.772025 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbba578-10f1-4859-855a-a67cea9bbf3e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:06.772306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbba578-10f1-4859-855a-a67cea9bbf3e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:06.772580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbba578-10f1-4859-855a-a67cea9bbf3e None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:06.773369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbba578-10f1-4859-855a-a67cea9bbf3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:06.773672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbba578-10f1-4859-855a-a67cea9bbf3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:06.775104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbba578-10f1-4859-855a-a67cea9bbf3e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:06.787930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 559/1099] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:40:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:06.793955 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbba578-10f1-4859-855a-a67cea9bbf3e None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:06.875150 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d9cb0-2d06-4d51-bc91-114c78f2d132 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:06.891226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:06.898825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:06.901413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbba578-10f1-4859-855a-a67cea9bbf3e None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:06.911981 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa0100-6a57-4815-89b9-81ec3085e813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:06.912780 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa0100-6a57-4815-89b9-81ec3085e813 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:06.913061 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa0100-6a57-4815-89b9-81ec3085e813 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:06.913338 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa0100-6a57-4815-89b9-81ec3085e813 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:06.923807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa0100-6a57-4815-89b9-81ec3085e813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-33aa0100-6a57-4815-89b9-81ec3085e813 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:40:06.931689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:06.932858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:06.933712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:06.933712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:06.933712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:06.933712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:06.933712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:40:06.933712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:40:06.933712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:40:06.933712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:40:06.933712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:40:06.933712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:40:06.933712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:06.933712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:40:06.933712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 541/1100] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:40:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:40:06.937411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86ecaf-b19f-4918-a671-3fae3afb9e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:06.938076 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86ecaf-b19f-4918-a671-3fae3afb9e61 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:06.938354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86ecaf-b19f-4918-a671-3fae3afb9e61 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:06.938631 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86ecaf-b19f-4918-a671-3fae3afb9e61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:06.938927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa0100-6a57-4815-89b9-81ec3085e813 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:06.939475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86ecaf-b19f-4918-a671-3fae3afb9e61 None admin] RBAC: Adding query filter params (role.id=952e60f3db5044afa8b3043155a1ccca, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=77812b356c974d2b80102217956169db) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:06.939818 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86ecaf-b19f-4918-a671-3fae3afb9e61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:06.941856 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86ecaf-b19f-4918-a671-3fae3afb9e61 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:06.962760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 560/1101] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:40:06 2022] GET /identity/v3/role_assignments?user.id=77812b356c974d2b80102217956169db&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=952e60f3db5044afa8b3043155a1ccca => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:07.054418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa0100-6a57-4815-89b9-81ec3085e813 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:07.096207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2953e-31d5-4138-b6e8-1328eb46aeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:07.096955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2953e-31d5-4138-b6e8-1328eb46aeb8 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:07.097351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2953e-31d5-4138-b6e8-1328eb46aeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:07.097706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2953e-31d5-4138-b6e8-1328eb46aeb8 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:07.098633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2953e-31d5-4138-b6e8-1328eb46aeb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:07.098940 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2953e-31d5-4138-b6e8-1328eb46aeb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:07.100368 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2953e-31d5-4138-b6e8-1328eb46aeb8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:07.111760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 542/1102] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:40:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:07.117520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86ecaf-b19f-4918-a671-3fae3afb9e61 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:07.230135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86ecaf-b19f-4918-a671-3fae3afb9e61 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:07.270234 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794730f-0ba5-4d8a-822b-dd7216cec155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:07.271110 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794730f-0ba5-4d8a-822b-dd7216cec155 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:07.271398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794730f-0ba5-4d8a-822b-dd7216cec155 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:07.271669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794730f-0ba5-4d8a-822b-dd7216cec155 None admin] PATH_INFO: `/v3/users/0212eef31bc14b29812686ec3bc9c55b` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:07.273067 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794730f-0ba5-4d8a-822b-dd7216cec155 None admin] RBAC: Authorizing `identity:get_user(user_id=0212eef31bc14b29812686ec3bc9c55b)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:07.274528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794730f-0ba5-4d8a-822b-dd7216cec155 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:07.276348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 561/1103] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:40:07 2022] GET /identity/v3/users/0212eef31bc14b29812686ec3bc9c55b => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:07.282189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2953e-31d5-4138-b6e8-1328eb46aeb8 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:07.332006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:07.390830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2953e-31d5-4138-b6e8-1328eb46aeb8 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:07.431445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121ddf1-5c1e-4c5d-a222-72a5c475c89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:07.432132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121ddf1-5c1e-4c5d-a222-72a5c475c89c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:07.432421 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121ddf1-5c1e-4c5d-a222-72a5c475c89c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:07.432698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121ddf1-5c1e-4c5d-a222-72a5c475c89c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:07.509774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121ddf1-5c1e-4c5d-a222-72a5c475c89c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:07.511242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121ddf1-5c1e-4c5d-a222-72a5c475c89c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:07.566511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:07.630237 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 543/1104] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:40:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:40:07.636223 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794730f-0ba5-4d8a-822b-dd7216cec155 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:07.664939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:07.748724 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794730f-0ba5-4d8a-822b-dd7216cec155 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:07.789262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45faa3-fc86-4126-a73a-07869c88433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:07.790138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45faa3-fc86-4126-a73a-07869c88433a None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:07.790418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45faa3-fc86-4126-a73a-07869c88433a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:07.790689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45faa3-fc86-4126-a73a-07869c88433a None admin] PATH_INFO: `/v3/projects/44d9127ed1f8417891c3f2106c514520` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:07.791927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45faa3-fc86-4126-a73a-07869c88433a None admin] RBAC: Authorizing `identity:get_project(project_id=44d9127ed1f8417891c3f2106c514520)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:07.793370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45faa3-fc86-4126-a73a-07869c88433a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:07.795132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 562/1105] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:40:07 2022] GET /identity/v3/projects/44d9127ed1f8417891c3f2106c514520 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:07.802277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121ddf1-5c1e-4c5d-a222-72a5c475c89c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:07.909379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121ddf1-5c1e-4c5d-a222-72a5c475c89c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:07.944446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fb4e9-7cd7-4307-8c6f-aca7dd1d6ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:07.945102 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fb4e9-7cd7-4307-8c6f-aca7dd1d6ff4 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:07.945380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fb4e9-7cd7-4307-8c6f-aca7dd1d6ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:07.945653 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fb4e9-7cd7-4307-8c6f-aca7dd1d6ff4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:07.946483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fb4e9-7cd7-4307-8c6f-aca7dd1d6ff4 None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=44d9127ed1f8417891c3f2106c514520, user.id=0212eef31bc14b29812686ec3bc9c55b) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:07.946790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fb4e9-7cd7-4307-8c6f-aca7dd1d6ff4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:07.948256 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fb4e9-7cd7-4307-8c6f-aca7dd1d6ff4 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:07.954265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 544/1106] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:40:07 2022] GET /identity/v3/role_assignments?user.id=0212eef31bc14b29812686ec3bc9c55b&scope.project.id=44d9127ed1f8417891c3f2106c514520&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:10.053778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6017c-75c1-419d-b4c4-371163385206 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:10.054320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6017c-75c1-419d-b4c4-371163385206 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:10.054320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6017c-75c1-419d-b4c4-371163385206 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:10.055750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 563/1107] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:10.061229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53850609-dbce-47c9-ad2d-710333e8c423 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:10.061672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53850609-dbce-47c9-ad2d-710333e8c423 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:10.061784 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53850609-dbce-47c9-ad2d-710333e8c423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:10.078837 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-53850609-dbce-47c9-ad2d-710333e8c423 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:10.080068 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-53850609-dbce-47c9-ad2d-710333e8c423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:10.082353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:10.150140 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 545/1108] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:10.176408 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6017c-75c1-419d-b4c4-371163385206 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:10.271580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:10.282779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6017c-75c1-419d-b4c4-371163385206 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:10.320898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8aee4-8970-4617-94aa-33ba71b653ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:10.321608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8aee4-8970-4617-94aa-33ba71b653ec None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:10.321887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8aee4-8970-4617-94aa-33ba71b653ec None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:10.322158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8aee4-8970-4617-94aa-33ba71b653ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:10.323359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8aee4-8970-4617-94aa-33ba71b653ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:10.324809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8aee4-8970-4617-94aa-33ba71b653ec None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:10.326458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 564/1109] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:40:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:10.333197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53850609-dbce-47c9-ad2d-710333e8c423 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:10.435831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53850609-dbce-47c9-ad2d-710333e8c423 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:10.470136 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1f63b-0c25-4276-bf53-8457036ff27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:10.470807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1f63b-0c25-4276-bf53-8457036ff27e None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:10.471086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1f63b-0c25-4276-bf53-8457036ff27e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:10.471365 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1f63b-0c25-4276-bf53-8457036ff27e None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:10.472131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1f63b-0c25-4276-bf53-8457036ff27e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:10.473669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1f63b-0c25-4276-bf53-8457036ff27e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:10.483352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:10.495671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 546/1110] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:40:10 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:12.574918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21b1b3-efc8-474f-b74a-614cec1780dc None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:12.575476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21b1b3-efc8-474f-b74a-614cec1780dc None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:12.575476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21b1b3-efc8-474f-b74a-614cec1780dc None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:12.576856 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 565/1111] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:12.582012 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0bf16-f023-4282-b4d9-ec455e50b8e6 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:12.582426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0bf16-f023-4282-b4d9-ec455e50b8e6 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:12.582568 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0bf16-f023-4282-b4d9-ec455e50b8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:12.599763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-47a0bf16-f023-4282-b4d9-ec455e50b8e6 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:12.600954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-47a0bf16-f023-4282-b4d9-ec455e50b8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:12.666052 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 547/1112] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:12.690565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21b1b3-efc8-474f-b74a-614cec1780dc None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:12.791078 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21b1b3-efc8-474f-b74a-614cec1780dc None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:12.828300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0fbe4-5c47-4398-9fa4-a18f111263d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:12.829145 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0fbe4-5c47-4398-9fa4-a18f111263d0 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:12.829428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0fbe4-5c47-4398-9fa4-a18f111263d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:12.829704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0fbe4-5c47-4398-9fa4-a18f111263d0 None admin] PATH_INFO: `/v3/roles/04597f92ac654fb4ac68dbb783d3dc31` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:12.831275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0fbe4-5c47-4398-9fa4-a18f111263d0 None admin] RBAC: Authorizing `identity:get_role(role_id=04597f92ac654fb4ac68dbb783d3dc31)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:12.833530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0fbe4-5c47-4398-9fa4-a18f111263d0 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:12.834928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 566/1113] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:40:12 2022] GET /identity/v3/roles/04597f92ac654fb4ac68dbb783d3dc31 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:12.840789 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0bf16-f023-4282-b4d9-ec455e50b8e6 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:12.939689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0bf16-f023-4282-b4d9-ec455e50b8e6 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:12.963534 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:12.975126 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cb465-45d5-43b0-a6f6-aad215e10c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:12.976041 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cb465-45d5-43b0-a6f6-aad215e10c68 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:12.976324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cb465-45d5-43b0-a6f6-aad215e10c68 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:12.976596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cb465-45d5-43b0-a6f6-aad215e10c68 None admin] PATH_INFO: `/v3/users/c55b45797fea4c11a5db4e8b77672754` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:12.990332 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cb465-45d5-43b0-a6f6-aad215e10c68 None admin] RBAC: Authorizing `identity:get_user(user_id=c55b45797fea4c11a5db4e8b77672754)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:12.991814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cb465-45d5-43b0-a6f6-aad215e10c68 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:12.993498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 548/1114] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:40:12 2022] GET /identity/v3/users/c55b45797fea4c11a5db4e8b77672754 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:12.999230 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0fbe4-5c47-4398-9fa4-a18f111263d0 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:13.027356 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:13.100726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0fbe4-5c47-4398-9fa4-a18f111263d0 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:13.136828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b3909-556e-4415-a7dd-b1f4e2105689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:13.137477 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b3909-556e-4415-a7dd-b1f4e2105689 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:13.137760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b3909-556e-4415-a7dd-b1f4e2105689 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:13.138036 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b3909-556e-4415-a7dd-b1f4e2105689 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:13.204533 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b3909-556e-4415-a7dd-b1f4e2105689 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:13.205899 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b3909-556e-4415-a7dd-b1f4e2105689 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:13.230913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:13.301901 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 567/1115] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:40:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:40:13.307835 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cb465-45d5-43b0-a6f6-aad215e10c68 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:13.415728 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cb465-45d5-43b0-a6f6-aad215e10c68 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:13.451681 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2795a53-30d0-44f7-a01f-53649962f2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:13.452576 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2795a53-30d0-44f7-a01f-53649962f2ff None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:13.452860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2795a53-30d0-44f7-a01f-53649962f2ff None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:13.453136 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2795a53-30d0-44f7-a01f-53649962f2ff None admin] PATH_INFO: `/v3/projects/44d9127ed1f8417891c3f2106c514520` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:13.454352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2795a53-30d0-44f7-a01f-53649962f2ff None admin] RBAC: Authorizing `identity:get_project(project_id=44d9127ed1f8417891c3f2106c514520)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:13.455875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2795a53-30d0-44f7-a01f-53649962f2ff None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:13.457905 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 549/1116] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:40:13 2022] GET /identity/v3/projects/44d9127ed1f8417891c3f2106c514520 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:13.466570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b3909-556e-4415-a7dd-b1f4e2105689 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:13.568802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b3909-556e-4415-a7dd-b1f4e2105689 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:13.594969 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:13.604424 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4027a-c1f7-4a9b-9e16-59f9292bc8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:13.605078 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4027a-c1f7-4a9b-9e16-59f9292bc8c6 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:13.605357 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4027a-c1f7-4a9b-9e16-59f9292bc8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:13.605631 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4027a-c1f7-4a9b-9e16-59f9292bc8c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:13.606460 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4027a-c1f7-4a9b-9e16-59f9292bc8c6 None admin] RBAC: Adding query filter params (role.id=04597f92ac654fb4ac68dbb783d3dc31, scope.project.id=44d9127ed1f8417891c3f2106c514520, user.id=c55b45797fea4c11a5db4e8b77672754) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:13.606774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4027a-c1f7-4a9b-9e16-59f9292bc8c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:13.608235 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4027a-c1f7-4a9b-9e16-59f9292bc8c6 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:13.613822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 568/1117] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:40:13 2022] GET /identity/v3/role_assignments?user.id=c55b45797fea4c11a5db4e8b77672754&scope.project.id=44d9127ed1f8417891c3f2106c514520&role.id=04597f92ac654fb4ac68dbb783d3dc31 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:15.629448 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80101f-3b8b-4397-a4b6-5a309d6c3f6c None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:15.629974 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80101f-3b8b-4397-a4b6-5a309d6c3f6c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:15.629974 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80101f-3b8b-4397-a4b6-5a309d6c3f6c None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:15.631353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 550/1118] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:15.636757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07663610-81e9-4478-ba7a-e16a1daa0829 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:15.637190 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07663610-81e9-4478-ba7a-e16a1daa0829 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:15.637317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07663610-81e9-4478-ba7a-e16a1daa0829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:15.655324 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-07663610-81e9-4478-ba7a-e16a1daa0829 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:15.656487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-07663610-81e9-4478-ba7a-e16a1daa0829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:15.720975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 569/1119] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:15.744778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80101f-3b8b-4397-a4b6-5a309d6c3f6c None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:15.793759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:15.842632 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80101f-3b8b-4397-a4b6-5a309d6c3f6c None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:15.880701 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0e0dc-2fb5-4050-8bb4-de6e3b975a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:15.881544 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0e0dc-2fb5-4050-8bb4-de6e3b975a4f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:15.881824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0e0dc-2fb5-4050-8bb4-de6e3b975a4f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:15.882096 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0e0dc-2fb5-4050-8bb4-de6e3b975a4f None admin] PATH_INFO: `/v3/roles/04597f92ac654fb4ac68dbb783d3dc31` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:15.883559 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0e0dc-2fb5-4050-8bb4-de6e3b975a4f None admin] RBAC: Authorizing `identity:get_role(role_id=04597f92ac654fb4ac68dbb783d3dc31)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:15.884940 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0e0dc-2fb5-4050-8bb4-de6e3b975a4f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:15.886303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 551/1120] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:40:15 2022] GET /identity/v3/roles/04597f92ac654fb4ac68dbb783d3dc31 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:15.892220 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07663610-81e9-4478-ba7a-e16a1daa0829 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:15.991330 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07663610-81e9-4478-ba7a-e16a1daa0829 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:16.010930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:16.025022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f0a7a-9315-46e4-80d4-a0d90ef0240f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:16.025892 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f0a7a-9315-46e4-80d4-a0d90ef0240f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.026176 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f0a7a-9315-46e4-80d4-a0d90ef0240f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.026452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f0a7a-9315-46e4-80d4-a0d90ef0240f None admin] PATH_INFO: `/v3/users/c55b45797fea4c11a5db4e8b77672754` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.027648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f0a7a-9315-46e4-80d4-a0d90ef0240f None admin] RBAC: Authorizing `identity:get_user(user_id=c55b45797fea4c11a5db4e8b77672754)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:16.029114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f0a7a-9315-46e4-80d4-a0d90ef0240f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:16.030780 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 570/1121] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:40:15 2022] GET /identity/v3/users/c55b45797fea4c11a5db4e8b77672754 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:16.037548 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0e0dc-2fb5-4050-8bb4-de6e3b975a4f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:16.135438 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0e0dc-2fb5-4050-8bb4-de6e3b975a4f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:16.151911 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:16.168807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da7703-f83e-40b2-8f33-2f7acb50e59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:16.169650 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da7703-f83e-40b2-8f33-2f7acb50e59b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.169928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da7703-f83e-40b2-8f33-2f7acb50e59b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.170196 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da7703-f83e-40b2-8f33-2f7acb50e59b None admin] PATH_INFO: `/v3/projects/44d9127ed1f8417891c3f2106c514520` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.171356 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da7703-f83e-40b2-8f33-2f7acb50e59b None admin] RBAC: Authorizing `identity:get_project(project_id=44d9127ed1f8417891c3f2106c514520)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:16.172801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da7703-f83e-40b2-8f33-2f7acb50e59b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:16.174495 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 552/1122] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:40:16 2022] GET /identity/v3/projects/44d9127ed1f8417891c3f2106c514520 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:16.181661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f0a7a-9315-46e4-80d4-a0d90ef0240f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:16.280869 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f0a7a-9315-46e4-80d4-a0d90ef0240f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:16.314175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e61b4-02a1-48e3-9465-b0fbbe18936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:16.315045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e61b4-02a1-48e3-9465-b0fbbe18936c None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:16.315324 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e61b4-02a1-48e3-9465-b0fbbe18936c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:16.315698 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e61b4-02a1-48e3-9465-b0fbbe18936c None admin] PATH_INFO: `/v3/projects/44d9127ed1f8417891c3f2106c514520/users/c55b45797fea4c11a5db4e8b77672754/roles/04597f92ac654fb4ac68dbb783d3dc31` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:16.317895 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e61b4-02a1-48e3-9465-b0fbbe18936c None admin] RBAC: Authorizing `identity:create_grant(project_id=44d9127ed1f8417891c3f2106c514520, user_id=c55b45797fea4c11a5db4e8b77672754, role_id=04597f92ac654fb4ac68dbb783d3dc31)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:16.319301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e61b4-02a1-48e3-9465-b0fbbe18936c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:16.332993 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 571/1123] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:40:16 2022] PUT /identity/v3/projects/44d9127ed1f8417891c3f2106c514520/users/c55b45797fea4c11a5db4e8b77672754/roles/04597f92ac654fb4ac68dbb783d3dc31 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:18.438929 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd1d59-5b38-4dc7-9902-4a6b38e29814 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.439452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd1d59-5b38-4dc7-9902-4a6b38e29814 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.439543 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd1d59-5b38-4dc7-9902-4a6b38e29814 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.441483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 553/1124] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:18.447093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ee4fc-0f43-4884-91a9-ffb420017eca None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.447551 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ee4fc-0f43-4884-91a9-ffb420017eca None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.447679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ee4fc-0f43-4884-91a9-ffb420017eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.464871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-654ee4fc-0f43-4884-91a9-ffb420017eca None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:18.466090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-654ee4fc-0f43-4884-91a9-ffb420017eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:18.513969 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:18.542916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 572/1125] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:18.566928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd1d59-5b38-4dc7-9902-4a6b38e29814 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.675376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd1d59-5b38-4dc7-9902-4a6b38e29814 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.711736 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524050b-f030-4729-b3a5-728ef3d93a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.712593 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524050b-f030-4729-b3a5-728ef3d93a78 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.712877 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524050b-f030-4729-b3a5-728ef3d93a78 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.713153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524050b-f030-4729-b3a5-728ef3d93a78 None admin] PATH_INFO: `/v3/roles/04597f92ac654fb4ac68dbb783d3dc31` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.714655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524050b-f030-4729-b3a5-728ef3d93a78 None admin] RBAC: Authorizing `identity:get_role(role_id=04597f92ac654fb4ac68dbb783d3dc31)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.716065 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524050b-f030-4729-b3a5-728ef3d93a78 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.717472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 554/1126] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:40:18 2022] GET /identity/v3/roles/04597f92ac654fb4ac68dbb783d3dc31 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:18.724548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ee4fc-0f43-4884-91a9-ffb420017eca None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.826433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ee4fc-0f43-4884-91a9-ffb420017eca None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:18.863968 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba75ea-31d9-4c0c-ac72-66fdb2d25b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:18.865982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba75ea-31d9-4c0c-ac72-66fdb2d25b00 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:18.865982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba75ea-31d9-4c0c-ac72-66fdb2d25b00 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:18.865982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba75ea-31d9-4c0c-ac72-66fdb2d25b00 None admin] PATH_INFO: `/v3/users/c55b45797fea4c11a5db4e8b77672754` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:18.867104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba75ea-31d9-4c0c-ac72-66fdb2d25b00 None admin] RBAC: Authorizing `identity:get_user(user_id=c55b45797fea4c11a5db4e8b77672754)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:18.868600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba75ea-31d9-4c0c-ac72-66fdb2d25b00 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:18.870279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 573/1127] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:40:18 2022] GET /identity/v3/users/c55b45797fea4c11a5db4e8b77672754 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:18.876102 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524050b-f030-4729-b3a5-728ef3d93a78 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:18.933129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:18.978296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524050b-f030-4729-b3a5-728ef3d93a78 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:19.013775 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f2324-e21a-41f0-8722-2a6c57b741b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:19.014466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f2324-e21a-41f0-8722-2a6c57b741b2 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:19.014745 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f2324-e21a-41f0-8722-2a6c57b741b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:19.015014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f2324-e21a-41f0-8722-2a6c57b741b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:19.093890 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f2324-e21a-41f0-8722-2a6c57b741b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:19.095353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f2324-e21a-41f0-8722-2a6c57b741b2 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:19.150841 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:19.196866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 555/1128] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:40:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:40:19.202840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba75ea-31d9-4c0c-ac72-66fdb2d25b00 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:19.252444 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:19.308822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba75ea-31d9-4c0c-ac72-66fdb2d25b00 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:19.345148 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e8f28-3e04-4791-a8a3-590536c3a47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:19.346008 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e8f28-3e04-4791-a8a3-590536c3a47c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:19.346289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e8f28-3e04-4791-a8a3-590536c3a47c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:19.346599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e8f28-3e04-4791-a8a3-590536c3a47c None admin] PATH_INFO: `/v3/projects/44d9127ed1f8417891c3f2106c514520` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:19.348733 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e8f28-3e04-4791-a8a3-590536c3a47c None admin] RBAC: Authorizing `identity:get_project(project_id=44d9127ed1f8417891c3f2106c514520)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:19.350175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e8f28-3e04-4791-a8a3-590536c3a47c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:19.351954 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 574/1129] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:40:19 2022] GET /identity/v3/projects/44d9127ed1f8417891c3f2106c514520 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:19.358406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f2324-e21a-41f0-8722-2a6c57b741b2 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:19.461082 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f2324-e21a-41f0-8722-2a6c57b741b2 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:19.495258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82003423-623f-49ac-be80-d6bcff431570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:19.495942 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82003423-623f-49ac-be80-d6bcff431570 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:19.496225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82003423-623f-49ac-be80-d6bcff431570 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:19.496498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82003423-623f-49ac-be80-d6bcff431570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:19.497343 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82003423-623f-49ac-be80-d6bcff431570 None admin] RBAC: Adding query filter params (role.id=04597f92ac654fb4ac68dbb783d3dc31, scope.project.id=44d9127ed1f8417891c3f2106c514520, user.id=c55b45797fea4c11a5db4e8b77672754) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:19.497655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82003423-623f-49ac-be80-d6bcff431570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:19.499112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82003423-623f-49ac-be80-d6bcff431570 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:19.505909 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 556/1130] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:40:19 2022] GET /identity/v3/role_assignments?user.id=c55b45797fea4c11a5db4e8b77672754&scope.project.id=44d9127ed1f8417891c3f2106c514520&role.id=04597f92ac654fb4ac68dbb783d3dc31 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:21.593556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484c7a7-841b-4e14-860f-620a3bbc75e3 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:21.594257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484c7a7-841b-4e14-860f-620a3bbc75e3 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:21.594257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484c7a7-841b-4e14-860f-620a3bbc75e3 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:21.595496 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 575/1131] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:21.601177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce6d1f-0afa-41b4-833d-af84f3bb7934 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:21.601690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce6d1f-0afa-41b4-833d-af84f3bb7934 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:21.601789 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce6d1f-0afa-41b4-833d-af84f3bb7934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:21.619172 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-bbce6d1f-0afa-41b4-833d-af84f3bb7934 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:21.620382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-bbce6d1f-0afa-41b4-833d-af84f3bb7934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:21.634669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:21.689006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 557/1132] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:21.714844 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484c7a7-841b-4e14-860f-620a3bbc75e3 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:21.820852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484c7a7-841b-4e14-860f-620a3bbc75e3 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:21.835610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:21.856679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb808f-3bab-42d6-8474-42703a5150c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:21.857475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb808f-3bab-42d6-8474-42703a5150c7 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:21.857759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb808f-3bab-42d6-8474-42703a5150c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:21.858036 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb808f-3bab-42d6-8474-42703a5150c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:21.859217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb808f-3bab-42d6-8474-42703a5150c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:21.860680 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feb808f-3bab-42d6-8474-42703a5150c7 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:21.863055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 576/1133] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:40:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:21.868897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce6d1f-0afa-41b4-833d-af84f3bb7934 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:21.973328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce6d1f-0afa-41b4-833d-af84f3bb7934 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:22.007844 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e48fb-b9dc-40c2-95f9-0572ea868f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:22.008511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e48fb-b9dc-40c2-95f9-0572ea868f13 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:22.008794 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e48fb-b9dc-40c2-95f9-0572ea868f13 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:22.009069 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e48fb-b9dc-40c2-95f9-0572ea868f13 None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:22.009795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e48fb-b9dc-40c2-95f9-0572ea868f13 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:22.011233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e48fb-b9dc-40c2-95f9-0572ea868f13 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:22.031367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:22.044970 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 558/1134] 10.222.0.60 () {60 vars in 1141 bytes} [Thu Jul 21 20:40:21 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:24.169506 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976b0f3-0d0b-4481-a54e-12a29755e149 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:24.170039 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976b0f3-0d0b-4481-a54e-12a29755e149 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:24.170039 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976b0f3-0d0b-4481-a54e-12a29755e149 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:24.171441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 577/1135] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:24.177142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b1c76-cb9a-4492-9bda-a2687748d9ed None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:24.177579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b1c76-cb9a-4492-9bda-a2687748d9ed None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:24.177682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b1c76-cb9a-4492-9bda-a2687748d9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:24.196241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-4a3b1c76-cb9a-4492-9bda-a2687748d9ed None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:24.197438 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-4a3b1c76-cb9a-4492-9bda-a2687748d9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:24.272958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 559/1136] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:24.297934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976b0f3-0d0b-4481-a54e-12a29755e149 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:24.403413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976b0f3-0d0b-4481-a54e-12a29755e149 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:24.439081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb27d4-7c5f-4512-999c-ed92ebf8233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:24.439811 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb27d4-7c5f-4512-999c-ed92ebf8233d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:24.440095 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb27d4-7c5f-4512-999c-ed92ebf8233d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:24.440394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb27d4-7c5f-4512-999c-ed92ebf8233d None admin] PATH_INFO: `/v3/domains/default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:24.441601 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb27d4-7c5f-4512-999c-ed92ebf8233d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:24.443024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb27d4-7c5f-4512-999c-ed92ebf8233d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:24.444712 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 578/1137] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:40:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:24.450535 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b1c76-cb9a-4492-9bda-a2687748d9ed None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:24.553162 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b1c76-cb9a-4492-9bda-a2687748d9ed None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:24.583924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:24.590362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91adb6-f390-4392-90f5-7b19b50126c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:24.591048 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91adb6-f390-4392-90f5-7b19b50126c8 None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:24.591329 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91adb6-f390-4392-90f5-7b19b50126c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:24.591605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91adb6-f390-4392-90f5-7b19b50126c8 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:24.592355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91adb6-f390-4392-90f5-7b19b50126c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:24.593923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91adb6-f390-4392-90f5-7b19b50126c8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:24.614833 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 560/1138] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:40:24 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:26.738438 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd334bc-ffee-418b-ac9b-880d262144c7 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:26.739046 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd334bc-ffee-418b-ac9b-880d262144c7 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:26.739046 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd334bc-ffee-418b-ac9b-880d262144c7 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:26.740297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 579/1139] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:26.745686 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fccb8-4b37-49e9-92fd-2739f8c92ded None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:26.746112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fccb8-4b37-49e9-92fd-2739f8c92ded None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:26.746239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fccb8-4b37-49e9-92fd-2739f8c92ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:26.763404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-8b5fccb8-4b37-49e9-92fd-2739f8c92ded None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:26.764642 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8b5fccb8-4b37-49e9-92fd-2739f8c92ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:26.835149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 561/1140] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:26.859576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd334bc-ffee-418b-ac9b-880d262144c7 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:26.912586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:26.965673 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd334bc-ffee-418b-ac9b-880d262144c7 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:27.001336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49103b5-90fc-4d7f-82ca-d472e340cd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:27.002021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49103b5-90fc-4d7f-82ca-d472e340cd8b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:27.002302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49103b5-90fc-4d7f-82ca-d472e340cd8b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:27.002610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49103b5-90fc-4d7f-82ca-d472e340cd8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:27.013825 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49103b5-90fc-4d7f-82ca-d472e340cd8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-b49103b5-90fc-4d7f-82ca-d472e340cd8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:40:27.021997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:27.023226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:27.024546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:27.024546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:27.024546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:27.024546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:27.024546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:40:27.024546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:40:27.024546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:40:27.024546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:40:27.024546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:40:27.024546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:40:27.024546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:27.024546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:40:27.024546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 580/1141] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:40:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:40:27.029393 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fccb8-4b37-49e9-92fd-2739f8c92ded None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:27.133381 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fccb8-4b37-49e9-92fd-2739f8c92ded None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:27.149892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:27.171154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629545c-560a-4498-8a7a-55fb3cde2876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:27.171869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629545c-560a-4498-8a7a-55fb3cde2876 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:27.172156 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629545c-560a-4498-8a7a-55fb3cde2876 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:27.172436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629545c-560a-4498-8a7a-55fb3cde2876 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:27.173191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629545c-560a-4498-8a7a-55fb3cde2876 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:27.173501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629545c-560a-4498-8a7a-55fb3cde2876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:27.175458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629545c-560a-4498-8a7a-55fb3cde2876 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:27.184089 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 562/1142] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:40:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:27.190663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49103b5-90fc-4d7f-82ca-d472e340cd8b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:27.291927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49103b5-90fc-4d7f-82ca-d472e340cd8b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:27.302519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:27.327466 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdf23c-0ece-4e30-9a84-39f1432b92cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:27.328385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdf23c-0ece-4e30-9a84-39f1432b92cd None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:27.328666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdf23c-0ece-4e30-9a84-39f1432b92cd None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:27.328942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdf23c-0ece-4e30-9a84-39f1432b92cd None admin] PATH_INFO: `/v3/users/940580e576f84030a0a86578df206dc2` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:27.343547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdf23c-0ece-4e30-9a84-39f1432b92cd None admin] RBAC: Authorizing `identity:get_user(user_id=940580e576f84030a0a86578df206dc2)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:27.345030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdf23c-0ece-4e30-9a84-39f1432b92cd None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:27.346798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 581/1143] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:40:27 2022] GET /identity/v3/users/940580e576f84030a0a86578df206dc2 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:27.353436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629545c-560a-4498-8a7a-55fb3cde2876 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:27.466369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629545c-560a-4498-8a7a-55fb3cde2876 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:27.501927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75816ef2-e935-42a4-a323-c0e8279e257b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:27.502618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816ef2-e935-42a4-a323-c0e8279e257b None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:27.502992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816ef2-e935-42a4-a323-c0e8279e257b None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:27.503549 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816ef2-e935-42a4-a323-c0e8279e257b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:27.552559 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:27.575982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75816ef2-e935-42a4-a323-c0e8279e257b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:27.577438 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75816ef2-e935-42a4-a323-c0e8279e257b None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:27.681071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 563/1144] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:40:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:40:27.687142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdf23c-0ece-4e30-9a84-39f1432b92cd None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:27.787328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdf23c-0ece-4e30-9a84-39f1432b92cd None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:27.822902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08eae3-42e9-439a-b8a0-1451208e9140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:27.823814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08eae3-42e9-439a-b8a0-1451208e9140 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:27.824101 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08eae3-42e9-439a-b8a0-1451208e9140 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:27.824381 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08eae3-42e9-439a-b8a0-1451208e9140 None admin] PATH_INFO: `/v3/projects/fe04850407d6477695ae39705b76b394` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:27.825576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08eae3-42e9-439a-b8a0-1451208e9140 None admin] RBAC: Authorizing `identity:get_project(project_id=fe04850407d6477695ae39705b76b394)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:27.827021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08eae3-42e9-439a-b8a0-1451208e9140 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:27.828785 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 582/1145] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:40:27 2022] GET /identity/v3/projects/fe04850407d6477695ae39705b76b394 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:27.835431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75816ef2-e935-42a4-a323-c0e8279e257b None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:27.914342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:27.936834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75816ef2-e935-42a4-a323-c0e8279e257b None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:27.973586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa41ced-310d-42bb-aec9-abde6d65e92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:27.974325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa41ced-310d-42bb-aec9-abde6d65e92e None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:27.974605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa41ced-310d-42bb-aec9-abde6d65e92e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:27.974876 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa41ced-310d-42bb-aec9-abde6d65e92e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:27.975766 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa41ced-310d-42bb-aec9-abde6d65e92e None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=fe04850407d6477695ae39705b76b394, user.id=940580e576f84030a0a86578df206dc2) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:27.976062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa41ced-310d-42bb-aec9-abde6d65e92e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:27.977504 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa41ced-310d-42bb-aec9-abde6d65e92e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:27.983913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 564/1146] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:40:27 2022] GET /identity/v3/role_assignments?user.id=940580e576f84030a0a86578df206dc2&scope.project.id=fe04850407d6477695ae39705b76b394&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:30.044344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0c9b5-a4f9-4229-82d4-1c670d76380f None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.044910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0c9b5-a4f9-4229-82d4-1c670d76380f None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.044910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0c9b5-a4f9-4229-82d4-1c670d76380f None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.046275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 583/1147] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:30.051807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e456a9b-709d-4687-aff0-981a479693d8 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.052289 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e456a9b-709d-4687-aff0-981a479693d8 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.052389 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e456a9b-709d-4687-aff0-981a479693d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.069593 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-2e456a9b-709d-4687-aff0-981a479693d8 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:30.070795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-2e456a9b-709d-4687-aff0-981a479693d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:30.139450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 565/1148] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:30.163329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0c9b5-a4f9-4229-82d4-1c670d76380f None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:30.200367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:30.273447 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0c9b5-a4f9-4229-82d4-1c670d76380f None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:30.311542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add70449-9d46-473e-a5cc-4aff15dc1bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:30.312294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add70449-9d46-473e-a5cc-4aff15dc1bcb None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.312576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add70449-9d46-473e-a5cc-4aff15dc1bcb None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.312852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add70449-9d46-473e-a5cc-4aff15dc1bcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.324596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add70449-9d46-473e-a5cc-4aff15dc1bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-add70449-9d46-473e-a5cc-4aff15dc1bcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:40:30.332898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:30.334833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:30.336442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:30.336442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:30.336442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:30.336442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:30.336442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:40:30.336442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:40:30.336442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:40:30.336442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:40:30.336442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:40:30.336442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:40:30.336442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:30.336442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:40:30.336442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 584/1149] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:40:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:40:30.340267 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e456a9b-709d-4687-aff0-981a479693d8 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:30.410692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:30.443197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e456a9b-709d-4687-aff0-981a479693d8 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:30.477754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ee80b-3c71-4668-a35c-2ad629369eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:30.478423 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ee80b-3c71-4668-a35c-2ad629369eca None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.478708 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ee80b-3c71-4668-a35c-2ad629369eca None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.479240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ee80b-3c71-4668-a35c-2ad629369eca None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.480041 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ee80b-3c71-4668-a35c-2ad629369eca None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:30.480354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ee80b-3c71-4668-a35c-2ad629369eca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:30.481737 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ee80b-3c71-4668-a35c-2ad629369eca None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:30.491030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 566/1150] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:40:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:30.497459 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add70449-9d46-473e-a5cc-4aff15dc1bcb None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:30.596893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:30.603198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add70449-9d46-473e-a5cc-4aff15dc1bcb None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:30.638932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74613f-9a4c-4239-97ad-0585026230cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:30.639852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74613f-9a4c-4239-97ad-0585026230cf None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.640182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74613f-9a4c-4239-97ad-0585026230cf None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.640456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74613f-9a4c-4239-97ad-0585026230cf None admin] PATH_INFO: `/v3/users/940580e576f84030a0a86578df206dc2` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.641667 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74613f-9a4c-4239-97ad-0585026230cf None admin] RBAC: Authorizing `identity:get_user(user_id=940580e576f84030a0a86578df206dc2)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:30.643148 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74613f-9a4c-4239-97ad-0585026230cf None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:30.645225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 585/1151] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:40:30 2022] GET /identity/v3/users/940580e576f84030a0a86578df206dc2 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:30.651466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ee80b-3c71-4668-a35c-2ad629369eca None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:30.758401 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ee80b-3c71-4668-a35c-2ad629369eca None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:30.794167 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf00463-4da8-488e-aa97-09002918bf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:30.795033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf00463-4da8-488e-aa97-09002918bf56 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.795320 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf00463-4da8-488e-aa97-09002918bf56 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.795599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf00463-4da8-488e-aa97-09002918bf56 None admin] PATH_INFO: `/v3/projects/fe04850407d6477695ae39705b76b394` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.796824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf00463-4da8-488e-aa97-09002918bf56 None admin] RBAC: Authorizing `identity:get_project(project_id=fe04850407d6477695ae39705b76b394)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:30.798259 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf00463-4da8-488e-aa97-09002918bf56 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:30.800709 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 567/1152] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:40:30 2022] GET /identity/v3/projects/fe04850407d6477695ae39705b76b394 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:30.806740 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74613f-9a4c-4239-97ad-0585026230cf None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:30.911345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74613f-9a4c-4239-97ad-0585026230cf None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:30.946276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072b1d3-7973-4731-b370-58d71677f5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:30.947162 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072b1d3-7973-4731-b370-58d71677f5ea None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:30.947446 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072b1d3-7973-4731-b370-58d71677f5ea None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:30.947734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072b1d3-7973-4731-b370-58d71677f5ea None admin] PATH_INFO: `/v3/projects/fe04850407d6477695ae39705b76b394/users/940580e576f84030a0a86578df206dc2/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:30.949717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072b1d3-7973-4731-b370-58d71677f5ea None admin] RBAC: Authorizing `identity:create_grant(project_id=fe04850407d6477695ae39705b76b394, user_id=940580e576f84030a0a86578df206dc2, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:30.951157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072b1d3-7973-4731-b370-58d71677f5ea None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:30.966921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 586/1153] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:40:30 2022] PUT /identity/v3/projects/fe04850407d6477695ae39705b76b394/users/940580e576f84030a0a86578df206dc2/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:32.997962 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dc5b4-b806-462c-a722-26ab297a92f0 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:32.998478 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dc5b4-b806-462c-a722-26ab297a92f0 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:32.998615 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dc5b4-b806-462c-a722-26ab297a92f0 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:32.999935 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 568/1154] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:33.005730 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453c256-70e4-41b1-a907-cb7e479da923 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.006124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453c256-70e4-41b1-a907-cb7e479da923 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.006294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453c256-70e4-41b1-a907-cb7e479da923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.022915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-9453c256-70e4-41b1-a907-cb7e479da923 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:33.024143 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9453c256-70e4-41b1-a907-cb7e479da923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:33.041248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:33.094368 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 587/1155] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:33.125278 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416dc5b4-b806-462c-a722-26ab297a92f0 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:33.128471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:33.243348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416dc5b4-b806-462c-a722-26ab297a92f0 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:33.278503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ef3d9-3068-440d-a3a1-303392a453c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:33.279215 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ef3d9-3068-440d-a3a1-303392a453c0 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.279496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ef3d9-3068-440d-a3a1-303392a453c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.279798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ef3d9-3068-440d-a3a1-303392a453c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.291695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ef3d9-3068-440d-a3a1-303392a453c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-ca2ef3d9-3068-440d-a3a1-303392a453c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:40:33.300009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:33.301124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:33.302141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:33.302141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:33.302141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:33.302141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:33.302141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:40:33.302141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:40:33.302141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:40:33.302141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:40:33.302141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:40:33.302141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:40:33.302141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:33.302141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:40:33.302141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 569/1156] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:40:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:40:33.308041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453c256-70e4-41b1-a907-cb7e479da923 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:33.422306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453c256-70e4-41b1-a907-cb7e479da923 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:33.465893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db02c61-bae1-463a-b922-2ffd076491a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:33.466558 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db02c61-bae1-463a-b922-2ffd076491a3 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.466838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db02c61-bae1-463a-b922-2ffd076491a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.467109 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db02c61-bae1-463a-b922-2ffd076491a3 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.467890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db02c61-bae1-463a-b922-2ffd076491a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:33.468194 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db02c61-bae1-463a-b922-2ffd076491a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:33.469754 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db02c61-bae1-463a-b922-2ffd076491a3 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:33.480064 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 588/1157] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:40:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:33.486679 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ef3d9-3068-440d-a3a1-303392a453c0 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:33.557701 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:33.609304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ef3d9-3068-440d-a3a1-303392a453c0 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:33.655492 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c27c3-c46e-4a4c-89f9-a13228ba4028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:33.656521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c27c3-c46e-4a4c-89f9-a13228ba4028 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.656811 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c27c3-c46e-4a4c-89f9-a13228ba4028 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.657084 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c27c3-c46e-4a4c-89f9-a13228ba4028 None admin] PATH_INFO: `/v3/users/940580e576f84030a0a86578df206dc2` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.658350 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c27c3-c46e-4a4c-89f9-a13228ba4028 None admin] RBAC: Authorizing `identity:get_user(user_id=940580e576f84030a0a86578df206dc2)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:33.661073 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c27c3-c46e-4a4c-89f9-a13228ba4028 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:33.663090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 570/1158] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:40:33 2022] GET /identity/v3/users/940580e576f84030a0a86578df206dc2 => generated 286 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:33.670354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db02c61-bae1-463a-b922-2ffd076491a3 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:33.693076 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:33.792588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db02c61-bae1-463a-b922-2ffd076491a3 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:33.838502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db659889-8fdc-47b1-9877-991da5289e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:33.839149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db659889-8fdc-47b1-9877-991da5289e72 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:33.839427 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db659889-8fdc-47b1-9877-991da5289e72 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:33.839703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db659889-8fdc-47b1-9877-991da5289e72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:33.923531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db659889-8fdc-47b1-9877-991da5289e72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:33.925059 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db659889-8fdc-47b1-9877-991da5289e72 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:33.944051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:34.050749 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 589/1159] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:40:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:40:34.057337 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c27c3-c46e-4a4c-89f9-a13228ba4028 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:34.163920 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c27c3-c46e-4a4c-89f9-a13228ba4028 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:34.191790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:34.198164 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b984c22-aa59-4432-899d-e78aa9a604b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:34.199039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b984c22-aa59-4432-899d-e78aa9a604b4 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:34.199321 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b984c22-aa59-4432-899d-e78aa9a604b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:34.199596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b984c22-aa59-4432-899d-e78aa9a604b4 None admin] PATH_INFO: `/v3/projects/fe04850407d6477695ae39705b76b394` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:34.200820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b984c22-aa59-4432-899d-e78aa9a604b4 None admin] RBAC: Authorizing `identity:get_project(project_id=fe04850407d6477695ae39705b76b394)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:34.202257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b984c22-aa59-4432-899d-e78aa9a604b4 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:34.204019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 571/1160] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:40:34 2022] GET /identity/v3/projects/fe04850407d6477695ae39705b76b394 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:34.210802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db659889-8fdc-47b1-9877-991da5289e72 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:34.311593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db659889-8fdc-47b1-9877-991da5289e72 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:34.328659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:34.347837 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0872cf-70cf-4f64-80b3-68d5c522e379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:34.348497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0872cf-70cf-4f64-80b3-68d5c522e379 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:34.348779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0872cf-70cf-4f64-80b3-68d5c522e379 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:34.349051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0872cf-70cf-4f64-80b3-68d5c522e379 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:34.349881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0872cf-70cf-4f64-80b3-68d5c522e379 None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=fe04850407d6477695ae39705b76b394, user.id=940580e576f84030a0a86578df206dc2) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:34.350326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0872cf-70cf-4f64-80b3-68d5c522e379 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:34.351628 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0872cf-70cf-4f64-80b3-68d5c522e379 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:34.358064 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 590/1161] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:40:34 2022] GET /identity/v3/role_assignments?user.id=940580e576f84030a0a86578df206dc2&scope.project.id=fe04850407d6477695ae39705b76b394&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:36.465265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a039294-4b25-465a-9e66-bb5cb39fc787 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:36.465812 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a039294-4b25-465a-9e66-bb5cb39fc787 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:36.465812 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a039294-4b25-465a-9e66-bb5cb39fc787 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:36.467190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 572/1162] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:36.472774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df104b0-3f30-4d1c-aee4-e51482d09d50 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:36.473209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df104b0-3f30-4d1c-aee4-e51482d09d50 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:36.473315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df104b0-3f30-4d1c-aee4-e51482d09d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:36.490867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-8df104b0-3f30-4d1c-aee4-e51482d09d50 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:36.492112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8df104b0-3f30-4d1c-aee4-e51482d09d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:36.563582 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 591/1163] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:36.586880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a039294-4b25-465a-9e66-bb5cb39fc787 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:36.692507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a039294-4b25-465a-9e66-bb5cb39fc787 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:36.726859 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcdea8-b3a9-4838-846f-f1047096c9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:36.727507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcdea8-b3a9-4838-846f-f1047096c9ba None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:36.727821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcdea8-b3a9-4838-846f-f1047096c9ba None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:36.728101 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcdea8-b3a9-4838-846f-f1047096c9ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:36.792324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:36.796356 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcdea8-b3a9-4838-846f-f1047096c9ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:36.797823 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcdea8-b3a9-4838-846f-f1047096c9ba None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:36.898901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 573/1164] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:40:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:40:36.905069 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df104b0-3f30-4d1c-aee4-e51482d09d50 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:37.012267 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df104b0-3f30-4d1c-aee4-e51482d09d50 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:37.029144 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:37.053967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe72b31-d840-4e4b-83cf-ddbc1c68f6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:37.054775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe72b31-d840-4e4b-83cf-ddbc1c68f6a6 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:37.055055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe72b31-d840-4e4b-83cf-ddbc1c68f6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:37.055330 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe72b31-d840-4e4b-83cf-ddbc1c68f6a6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:37.062186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe72b31-d840-4e4b-83cf-ddbc1c68f6a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:37.063628 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe72b31-d840-4e4b-83cf-ddbc1c68f6a6 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-ebe72b31-d840-4e4b-83cf-ddbc1c68f6a6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:37.078509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:37.079801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:37.081386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:37.081386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:37.081386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:37.081386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:37.081386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:37.081386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:37.081386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:37.081386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:37.081386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:37.081386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:37.081386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 21 20:40:37.081386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:40:37.081386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 20:40:37.081386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:40:37.081386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 592/1165] 10.222.0.60 () {56 vars in 1117 bytes} [Thu Jul 21 20:40:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:40:37.086560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcdea8-b3a9-4838-846f-f1047096c9ba None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:37.186452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:37.191642 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcdea8-b3a9-4838-846f-f1047096c9ba None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:37.228355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b675f-0173-4edf-a351-9ac8441d6f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:37.229283 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b675f-0173-4edf-a351-9ac8441d6f32 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:37.229570 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b675f-0173-4edf-a351-9ac8441d6f32 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:37.229842 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b675f-0173-4edf-a351-9ac8441d6f32 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:37.230570 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b675f-0173-4edf-a351-9ac8441d6f32 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:37.230870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b675f-0173-4edf-a351-9ac8441d6f32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:37.232348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b675f-0173-4edf-a351-9ac8441d6f32 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:37.240875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 574/1166] 10.222.0.60 () {56 vars in 1115 bytes} [Thu Jul 21 20:40:37 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:39.264038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59392339-58c5-47df-9d9d-8a75b7dbf840 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:39.264572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59392339-58c5-47df-9d9d-8a75b7dbf840 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:39.264572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59392339-58c5-47df-9d9d-8a75b7dbf840 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:39.265958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 593/1167] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:39.271832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88aa3d6-406f-4f01-bed7-a96124ca9a84 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:39.272228 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88aa3d6-406f-4f01-bed7-a96124ca9a84 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:39.272393 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88aa3d6-406f-4f01-bed7-a96124ca9a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:39.289584 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d88aa3d6-406f-4f01-bed7-a96124ca9a84 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:39.290770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d88aa3d6-406f-4f01-bed7-a96124ca9a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:39.358437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 575/1168] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:39.382212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59392339-58c5-47df-9d9d-8a75b7dbf840 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:39.486125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59392339-58c5-47df-9d9d-8a75b7dbf840 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:39.519919 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16419fe-952b-427b-95f5-1045d276db0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:39.520510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16419fe-952b-427b-95f5-1045d276db0a None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:39.520797 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16419fe-952b-427b-95f5-1045d276db0a None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:39.521077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16419fe-952b-427b-95f5-1045d276db0a None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:39.521822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16419fe-952b-427b-95f5-1045d276db0a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:39.523193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16419fe-952b-427b-95f5-1045d276db0a None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:39.548618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 594/1169] 10.222.0.60 () {60 vars in 1138 bytes} [Thu Jul 21 20:40:39 2022] POST /identity/v3/domains => generated 267 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:41.679822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db986c26-8a0a-43e1-943f-93ed9a60aec9 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:41.680345 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db986c26-8a0a-43e1-943f-93ed9a60aec9 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:41.680345 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db986c26-8a0a-43e1-943f-93ed9a60aec9 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:41.681681 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 576/1170] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:41.686771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a64d15-3a04-45e6-9ee8-d8d48ce747ef None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:41.687140 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a64d15-3a04-45e6-9ee8-d8d48ce747ef None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:41.687333 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a64d15-3a04-45e6-9ee8-d8d48ce747ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:41.705074 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-82a64d15-3a04-45e6-9ee8-d8d48ce747ef None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:41.706314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-82a64d15-3a04-45e6-9ee8-d8d48ce747ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:41.729552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:41.774344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 595/1171] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:41.799444 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db986c26-8a0a-43e1-943f-93ed9a60aec9 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:41.889626 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:41.899908 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db986c26-8a0a-43e1-943f-93ed9a60aec9 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:41.933640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee50cae-1f22-410d-8986-d4535aaa3fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:41.934421 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee50cae-1f22-410d-8986-d4535aaa3fa3 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:41.934705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee50cae-1f22-410d-8986-d4535aaa3fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:41.934979 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee50cae-1f22-410d-8986-d4535aaa3fa3 None admin] PATH_INFO: `/v3/domains/ee3966225a814a34af90d77ab017b3d0` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:41.946018 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee50cae-1f22-410d-8986-d4535aaa3fa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ee3966225a814a34af90d77ab017b3d0)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:41.947458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee50cae-1f22-410d-8986-d4535aaa3fa3 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:41.949164 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 577/1172] 10.222.0.60 () {56 vars in 1183 bytes} [Thu Jul 21 20:40:41 2022] GET /identity/v3/domains/ee3966225a814a34af90d77ab017b3d0 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:41.955280 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a64d15-3a04-45e6-9ee8-d8d48ce747ef None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:42.060470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a64d15-3a04-45e6-9ee8-d8d48ce747ef None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:42.095473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138118de-ec11-4a94-9b28-34e030ee78ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:42.096698 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138118de-ec11-4a94-9b28-34e030ee78ad None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:42.096980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138118de-ec11-4a94-9b28-34e030ee78ad None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:42.097300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138118de-ec11-4a94-9b28-34e030ee78ad None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:42.098077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138118de-ec11-4a94-9b28-34e030ee78ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:42.099533 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138118de-ec11-4a94-9b28-34e030ee78ad None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:42.127804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 596/1173] 10.222.0.60 () {60 vars in 1141 bytes} [Thu Jul 21 20:40:41 2022] POST /identity/v3/projects => generated 361 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:44.232841 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f670a9-687a-42e9-bb7b-72bb7b2b6111 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:44.233397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f670a9-687a-42e9-bb7b-72bb7b2b6111 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:44.233397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f670a9-687a-42e9-bb7b-72bb7b2b6111 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:44.234763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 578/1174] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:44.240679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34c6cd-cf51-4482-95ac-cee9e824053e None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:44.241060 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34c6cd-cf51-4482-95ac-cee9e824053e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:44.241247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34c6cd-cf51-4482-95ac-cee9e824053e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:44.258384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-8b34c6cd-cf51-4482-95ac-cee9e824053e None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:44.259571 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8b34c6cd-cf51-4482-95ac-cee9e824053e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:44.266773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:44.325317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 597/1175] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:44.350436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f670a9-687a-42e9-bb7b-72bb7b2b6111 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:44.450970 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f670a9-687a-42e9-bb7b-72bb7b2b6111 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:44.485627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2916e2-df78-4530-8654-6d4cea385be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:44.487104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2916e2-df78-4530-8654-6d4cea385be0 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:44.487610 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2916e2-df78-4530-8654-6d4cea385be0 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:44.487954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2916e2-df78-4530-8654-6d4cea385be0 None admin] PATH_INFO: `/v3/domains/ee3966225a814a34af90d77ab017b3d0` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:44.489557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2916e2-df78-4530-8654-6d4cea385be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ee3966225a814a34af90d77ab017b3d0)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:44.490992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2916e2-df78-4530-8654-6d4cea385be0 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:44.494042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 579/1176] 10.222.0.60 () {56 vars in 1183 bytes} [Thu Jul 21 20:40:44 2022] GET /identity/v3/domains/ee3966225a814a34af90d77ab017b3d0 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:44.499838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34c6cd-cf51-4482-95ac-cee9e824053e None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:44.596690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34c6cd-cf51-4482-95ac-cee9e824053e None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:44.631425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34177a-624c-4339-a88e-8d32d0a80b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:44.632109 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34177a-624c-4339-a88e-8d32d0a80b5d None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:44.632388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34177a-624c-4339-a88e-8d32d0a80b5d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:44.632659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34177a-624c-4339-a88e-8d32d0a80b5d None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:44.633370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34177a-624c-4339-a88e-8d32d0a80b5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:44.634782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34177a-624c-4339-a88e-8d32d0a80b5d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:44.649544 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:44.659770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 598/1177] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:40:44 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:40:46.731537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2642e-4592-4bdb-bbdb-676458a86f0d None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:46.732089 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2642e-4592-4bdb-bbdb-676458a86f0d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:46.732089 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2642e-4592-4bdb-bbdb-676458a86f0d None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:46.733417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 580/1178] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:46.739258 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478faf5-6d6b-4b89-94ed-fb700254c307 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:46.739703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478faf5-6d6b-4b89-94ed-fb700254c307 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:46.739842 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478faf5-6d6b-4b89-94ed-fb700254c307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:46.756992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-5478faf5-6d6b-4b89-94ed-fb700254c307 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:46.758207 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-5478faf5-6d6b-4b89-94ed-fb700254c307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:46.824182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 599/1179] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:40:46.849067 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2642e-4592-4bdb-bbdb-676458a86f0d None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:46.870922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:46.951454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2642e-4592-4bdb-bbdb-676458a86f0d None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:46.984651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de1380-9165-47c1-826d-ec795737605c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:46.985366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de1380-9165-47c1-826d-ec795737605c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:46.985648 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de1380-9165-47c1-826d-ec795737605c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:46.985920 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de1380-9165-47c1-826d-ec795737605c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:46.996627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de1380-9165-47c1-826d-ec795737605c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-44de1380-9165-47c1-826d-ec795737605c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:40:47.005112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:47.006451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:47.007682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:47.007682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:47.007682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:47.007682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:47.007682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:40:47.007682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:40:47.007682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:40:47.007682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:40:47.007682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:40:47.007682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:40:47.007682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:47.007682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:40:47.007682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 581/1180] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:40:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:40:47.012559 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5478faf5-6d6b-4b89-94ed-fb700254c307 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.112298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5478faf5-6d6b-4b89-94ed-fb700254c307 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:47.138140 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:47.146077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35d0dd-0ee0-49f6-af50-612380350d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:47.146736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35d0dd-0ee0-49f6-af50-612380350d45 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.147050 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35d0dd-0ee0-49f6-af50-612380350d45 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.147345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35d0dd-0ee0-49f6-af50-612380350d45 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.148212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35d0dd-0ee0-49f6-af50-612380350d45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:47.148520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35d0dd-0ee0-49f6-af50-612380350d45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:47.149883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35d0dd-0ee0-49f6-af50-612380350d45 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:47.157870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 600/1181] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:40:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:47.163934 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de1380-9165-47c1-826d-ec795737605c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.245319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:47.263943 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de1380-9165-47c1-826d-ec795737605c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:47.296508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad84c4-1bc5-49a2-a26f-bbb8e7e3f082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:47.297361 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad84c4-1bc5-49a2-a26f-bbb8e7e3f082 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.297635 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad84c4-1bc5-49a2-a26f-bbb8e7e3f082 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.297901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad84c4-1bc5-49a2-a26f-bbb8e7e3f082 None admin] PATH_INFO: `/v3/users/3056015ef17c4a8bb751e0ef164e594b` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.310620 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad84c4-1bc5-49a2-a26f-bbb8e7e3f082 None admin] RBAC: Authorizing `identity:get_user(user_id=3056015ef17c4a8bb751e0ef164e594b)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:47.312069 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad84c4-1bc5-49a2-a26f-bbb8e7e3f082 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:47.313715 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 582/1182] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:40:47 2022] GET /identity/v3/users/3056015ef17c4a8bb751e0ef164e594b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:47.320120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35d0dd-0ee0-49f6-af50-612380350d45 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.420041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35d0dd-0ee0-49f6-af50-612380350d45 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:47.453584 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadb61c-a702-44de-8f75-4392230a2d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:47.454227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadb61c-a702-44de-8f75-4392230a2d1b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.454506 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadb61c-a702-44de-8f75-4392230a2d1b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.454779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadb61c-a702-44de-8f75-4392230a2d1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.511434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:47.522465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadb61c-a702-44de-8f75-4392230a2d1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:47.523918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadb61c-a702-44de-8f75-4392230a2d1b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:47.623195 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 601/1183] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:40:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:40:47.629755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad84c4-1bc5-49a2-a26f-bbb8e7e3f082 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.732688 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad84c4-1bc5-49a2-a26f-bbb8e7e3f082 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:47.766125 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777a4bd-a049-4b0c-a367-6f5d8f00b1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:47.766904 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777a4bd-a049-4b0c-a367-6f5d8f00b1c3 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.767183 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777a4bd-a049-4b0c-a367-6f5d8f00b1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.767456 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777a4bd-a049-4b0c-a367-6f5d8f00b1c3 None admin] PATH_INFO: `/v3/projects/a7fcf23cc6a5480894b021605c9ea7e0` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.768761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777a4bd-a049-4b0c-a367-6f5d8f00b1c3 None admin] RBAC: Authorizing `identity:get_project(project_id=a7fcf23cc6a5480894b021605c9ea7e0)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:47.770179 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777a4bd-a049-4b0c-a367-6f5d8f00b1c3 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:47.771898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 583/1184] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:40:47 2022] GET /identity/v3/projects/a7fcf23cc6a5480894b021605c9ea7e0 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:47.778101 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadb61c-a702-44de-8f75-4392230a2d1b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:47.866454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:47.875946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadb61c-a702-44de-8f75-4392230a2d1b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:47.908557 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976afc23-8ef6-46f8-85f8-f62ea33361db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:47.909186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976afc23-8ef6-46f8-85f8-f62ea33361db None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:47.909465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976afc23-8ef6-46f8-85f8-f62ea33361db None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:47.909736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976afc23-8ef6-46f8-85f8-f62ea33361db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:47.910550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976afc23-8ef6-46f8-85f8-f62ea33361db None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=a7fcf23cc6a5480894b021605c9ea7e0, user.id=3056015ef17c4a8bb751e0ef164e594b) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:47.910853 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976afc23-8ef6-46f8-85f8-f62ea33361db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:47.912306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976afc23-8ef6-46f8-85f8-f62ea33361db None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:47.917587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 602/1185] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:40:47 2022] GET /identity/v3/role_assignments?user.id=3056015ef17c4a8bb751e0ef164e594b&scope.project.id=a7fcf23cc6a5480894b021605c9ea7e0&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:49.948850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372f5c2-a67d-42c8-9610-4e171400b25e None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:49.949376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372f5c2-a67d-42c8-9610-4e171400b25e None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:49.949376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372f5c2-a67d-42c8-9610-4e171400b25e None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:49.950773 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 584/1186] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:49.956535 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077c553-ad61-44a2-851b-5c9847a090e9 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:49.956976 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077c553-ad61-44a2-851b-5c9847a090e9 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:49.957114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077c553-ad61-44a2-851b-5c9847a090e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:49.975611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-6077c553-ad61-44a2-851b-5c9847a090e9 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:49.977189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6077c553-ad61-44a2-851b-5c9847a090e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:50.048035 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 603/1187] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:50.070687 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372f5c2-a67d-42c8-9610-4e171400b25e None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:50.079610 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:50.174488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372f5c2-a67d-42c8-9610-4e171400b25e None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:50.208003 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee5db3-f3fa-4129-87e0-396ac4e4fb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:50.208718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee5db3-f3fa-4129-87e0-396ac4e4fb2c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:50.209005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee5db3-f3fa-4129-87e0-396ac4e4fb2c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:50.209279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee5db3-f3fa-4129-87e0-396ac4e4fb2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:50.219138 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee5db3-f3fa-4129-87e0-396ac4e4fb2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-5bee5db3-f3fa-4129-87e0-396ac4e4fb2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:40:50.227260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:50.228349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:50.229484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:50.229484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:50.229484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:50.229484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:50.229484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:40:50.229484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:40:50.229484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:40:50.229484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:40:50.229484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:40:50.229484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:40:50.229484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:50.229484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:40:50.229484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 585/1188] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:40:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:40:50.235152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077c553-ad61-44a2-851b-5c9847a090e9 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:50.319490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:50.336964 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077c553-ad61-44a2-851b-5c9847a090e9 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:50.369423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fea8d0-dd59-40b3-8d17-150c3b95219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:50.370074 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fea8d0-dd59-40b3-8d17-150c3b95219f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:50.370352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fea8d0-dd59-40b3-8d17-150c3b95219f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:50.370625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fea8d0-dd59-40b3-8d17-150c3b95219f None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:50.371360 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fea8d0-dd59-40b3-8d17-150c3b95219f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:50.371660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fea8d0-dd59-40b3-8d17-150c3b95219f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:50.373038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fea8d0-dd59-40b3-8d17-150c3b95219f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:50.381413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 604/1189] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:40:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:50.387756 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee5db3-f3fa-4129-87e0-396ac4e4fb2c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:50.453697 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:50.489688 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee5db3-f3fa-4129-87e0-396ac4e4fb2c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:50.523160 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350b4e5-321d-406b-b0b0-1fa0674b5ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:50.524049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350b4e5-321d-406b-b0b0-1fa0674b5ee4 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:50.524335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350b4e5-321d-406b-b0b0-1fa0674b5ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:50.524611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350b4e5-321d-406b-b0b0-1fa0674b5ee4 None admin] PATH_INFO: `/v3/users/3056015ef17c4a8bb751e0ef164e594b` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:50.525809 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350b4e5-321d-406b-b0b0-1fa0674b5ee4 None admin] RBAC: Authorizing `identity:get_user(user_id=3056015ef17c4a8bb751e0ef164e594b)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:50.527237 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350b4e5-321d-406b-b0b0-1fa0674b5ee4 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:50.528913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 586/1190] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:40:50 2022] GET /identity/v3/users/3056015ef17c4a8bb751e0ef164e594b => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:50.535686 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fea8d0-dd59-40b3-8d17-150c3b95219f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:50.634659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fea8d0-dd59-40b3-8d17-150c3b95219f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:50.667885 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea4f32f-0187-46da-829d-7c336279c398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:50.668749 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4f32f-0187-46da-829d-7c336279c398 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:50.669033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4f32f-0187-46da-829d-7c336279c398 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:50.669308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4f32f-0187-46da-829d-7c336279c398 None admin] PATH_INFO: `/v3/projects/a7fcf23cc6a5480894b021605c9ea7e0` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:50.670497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4f32f-0187-46da-829d-7c336279c398 None admin] RBAC: Authorizing `identity:get_project(project_id=a7fcf23cc6a5480894b021605c9ea7e0)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:50.671956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea4f32f-0187-46da-829d-7c336279c398 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:50.673683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 605/1191] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:40:50 2022] GET /identity/v3/projects/a7fcf23cc6a5480894b021605c9ea7e0 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:40:50.680595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350b4e5-321d-406b-b0b0-1fa0674b5ee4 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:50.787379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350b4e5-321d-406b-b0b0-1fa0674b5ee4 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:50.820359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a407c7-b3f4-4cc7-a134-71575ad771f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:50.821241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a407c7-b3f4-4cc7-a134-71575ad771f2 None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:50.821521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a407c7-b3f4-4cc7-a134-71575ad771f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:50.821801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a407c7-b3f4-4cc7-a134-71575ad771f2 None admin] PATH_INFO: `/v3/projects/a7fcf23cc6a5480894b021605c9ea7e0/users/3056015ef17c4a8bb751e0ef164e594b/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:50.823770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a407c7-b3f4-4cc7-a134-71575ad771f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a7fcf23cc6a5480894b021605c9ea7e0, user_id=3056015ef17c4a8bb751e0ef164e594b, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:50.825197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a407c7-b3f4-4cc7-a134-71575ad771f2 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:50.827382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:50.839198 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 587/1192] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:40:50 2022] PUT /identity/v3/projects/a7fcf23cc6a5480894b021605c9ea7e0/users/3056015ef17c4a8bb751e0ef164e594b/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:40:52.859995 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d2c11-0eb3-402d-ad84-fe36d6f1a63e None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:52.860570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d2c11-0eb3-402d-ad84-fe36d6f1a63e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:52.860570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d2c11-0eb3-402d-ad84-fe36d6f1a63e None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:52.861941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 606/1193] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:40:52.867104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77872d54-72eb-4332-97d6-13170719ad2f None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:52.867549 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77872d54-72eb-4332-97d6-13170719ad2f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:52.867651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77872d54-72eb-4332-97d6-13170719ad2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:52.884783 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-77872d54-72eb-4332-97d6-13170719ad2f None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:40:52.885960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-77872d54-72eb-4332-97d6-13170719ad2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:40:52.953596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 588/1194] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:40:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:40:52.976810 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d2c11-0eb3-402d-ad84-fe36d6f1a63e None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:52.994039 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:53.077202 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d2c11-0eb3-402d-ad84-fe36d6f1a63e None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:53.110294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894a469-721d-4de9-8d5c-0459f05b5e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:53.111001 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894a469-721d-4de9-8d5c-0459f05b5e87 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:53.111809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894a469-721d-4de9-8d5c-0459f05b5e87 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:53.112193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894a469-721d-4de9-8d5c-0459f05b5e87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:53.121852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894a469-721d-4de9-8d5c-0459f05b5e87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-b894a469-721d-4de9-8d5c-0459f05b5e87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:40:53.129362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:40:53.130596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:40:53.131742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:40:53.131742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:40:53.131742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:40:53.131742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:40:53.131742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:40:53.131742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:40:53.131742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:40:53.131742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:40:53.131742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:40:53.131742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:40:53.131742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:40:53.131742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:40:53.131742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 607/1195] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:40:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:40:53.136691 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77872d54-72eb-4332-97d6-13170719ad2f None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:53.232981 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77872d54-72eb-4332-97d6-13170719ad2f None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:53.255457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:53.267094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2540bec-f00c-41c3-aafe-30e7ec16cd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:53.268296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2540bec-f00c-41c3-aafe-30e7ec16cd14 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:53.268583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2540bec-f00c-41c3-aafe-30e7ec16cd14 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:53.268859 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2540bec-f00c-41c3-aafe-30e7ec16cd14 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:53.269604 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2540bec-f00c-41c3-aafe-30e7ec16cd14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:53.269912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2540bec-f00c-41c3-aafe-30e7ec16cd14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:53.271276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2540bec-f00c-41c3-aafe-30e7ec16cd14 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:53.279254 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 589/1196] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:40:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:53.285860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894a469-721d-4de9-8d5c-0459f05b5e87 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:53.360184 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:53.385193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894a469-721d-4de9-8d5c-0459f05b5e87 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:53.418287 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c9bcf-973d-40af-a288-d0401dff3877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:53.419158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c9bcf-973d-40af-a288-d0401dff3877 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:53.419442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c9bcf-973d-40af-a288-d0401dff3877 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:53.419747 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c9bcf-973d-40af-a288-d0401dff3877 None admin] PATH_INFO: `/v3/users/3056015ef17c4a8bb751e0ef164e594b` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:53.420927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c9bcf-973d-40af-a288-d0401dff3877 None admin] RBAC: Authorizing `identity:get_user(user_id=3056015ef17c4a8bb751e0ef164e594b)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:53.422352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c9bcf-973d-40af-a288-d0401dff3877 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:53.424301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 608/1197] 10.222.0.60 () {56 vars in 1177 bytes} [Thu Jul 21 20:40:53 2022] GET /identity/v3/users/3056015ef17c4a8bb751e0ef164e594b => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:53.429796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2540bec-f00c-41c3-aafe-30e7ec16cd14 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:53.531677 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2540bec-f00c-41c3-aafe-30e7ec16cd14 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:53.565679 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a63b39-5c7e-4967-8583-ea0924719b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:53.566327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a63b39-5c7e-4967-8583-ea0924719b4c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:53.566608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a63b39-5c7e-4967-8583-ea0924719b4c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:53.566880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a63b39-5c7e-4967-8583-ea0924719b4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:53.620442 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:53.634605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a63b39-5c7e-4967-8583-ea0924719b4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:53.636122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a63b39-5c7e-4967-8583-ea0924719b4c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:53.733375 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 590/1198] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:40:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:40:53.739907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c9bcf-973d-40af-a288-d0401dff3877 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:53.837610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c9bcf-973d-40af-a288-d0401dff3877 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:53.870903 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cab08e-e552-499b-9c7a-4891f557c425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:53.871775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cab08e-e552-499b-9c7a-4891f557c425 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:53.872068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cab08e-e552-499b-9c7a-4891f557c425 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:53.872346 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cab08e-e552-499b-9c7a-4891f557c425 None admin] PATH_INFO: `/v3/projects/a7fcf23cc6a5480894b021605c9ea7e0` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:53.873532 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cab08e-e552-499b-9c7a-4891f557c425 None admin] RBAC: Authorizing `identity:get_project(project_id=a7fcf23cc6a5480894b021605c9ea7e0)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:53.874957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cab08e-e552-499b-9c7a-4891f557c425 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:53.876689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 609/1199] 10.222.0.60 () {56 vars in 1186 bytes} [Thu Jul 21 20:40:53 2022] GET /identity/v3/projects/a7fcf23cc6a5480894b021605c9ea7e0 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:40:53.883564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a63b39-5c7e-4967-8583-ea0924719b4c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:40:53.964621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:40:53.980271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a63b39-5c7e-4967-8583-ea0924719b4c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:40:54.014787 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eecd18-56c6-4314-90e6-3fb230389053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:40:54.015550 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eecd18-56c6-4314-90e6-3fb230389053 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:40:54.015872 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eecd18-56c6-4314-90e6-3fb230389053 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:40:54.016154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eecd18-56c6-4314-90e6-3fb230389053 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:40:54.016993 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eecd18-56c6-4314-90e6-3fb230389053 None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=a7fcf23cc6a5480894b021605c9ea7e0, user.id=3056015ef17c4a8bb751e0ef164e594b) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:40:54.017306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eecd18-56c6-4314-90e6-3fb230389053 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:40:54.018775 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eecd18-56c6-4314-90e6-3fb230389053 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:40:54.024695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 591/1200] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:40:53 2022] GET /identity/v3/role_assignments?user.id=3056015ef17c4a8bb751e0ef164e594b&scope.project.id=a7fcf23cc6a5480894b021605c9ea7e0&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:41:03.866957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=95 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:04.021978 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=25 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:50.272228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611f01e-1a44-4dd8-b310-de2bb316b4fa None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:50.272228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611f01e-1a44-4dd8-b310-de2bb316b4fa None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:50.272228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611f01e-1a44-4dd8-b310-de2bb316b4fa None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:50.273403 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 610/1201] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:41:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:41:50.279450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc664bc-dfdf-4afd-b5be-9fd27bc491fc None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:50.280062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc664bc-dfdf-4afd-b5be-9fd27bc491fc None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:50.280062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc664bc-dfdf-4afd-b5be-9fd27bc491fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:50.301398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-1bc664bc-dfdf-4afd-b5be-9fd27bc491fc None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:50.301398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1bc664bc-dfdf-4afd-b5be-9fd27bc491fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:50.398206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 592/1202] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:41:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:41:50.423321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611f01e-1a44-4dd8-b310-de2bb316b4fa None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:50.559668 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611f01e-1a44-4dd8-b310-de2bb316b4fa None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:50.611850 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2379b3-5286-453c-9175-a7647fd8ce0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:50.613068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2379b3-5286-453c-9175-a7647fd8ce0f None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:50.613390 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2379b3-5286-453c-9175-a7647fd8ce0f None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:50.613746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2379b3-5286-453c-9175-a7647fd8ce0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:50.627383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2379b3-5286-453c-9175-a7647fd8ce0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:50.630618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2379b3-5286-453c-9175-a7647fd8ce0f None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:50.656813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-7e2379b3-5286-453c-9175-a7647fd8ce0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:41:50.656813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:41:50.656813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:41:50.656813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:41:50.656813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:41:50.656813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:41:50.656813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:41:50.656813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:41:50.656813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:41:50.656813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:41:50.656813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:41:50.656813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:41:50.656813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:41:50.656813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:41:50.656813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:41:50.657851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:41:50.658846 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:41:50.658846 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:41:50.658846 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:41:50.658846 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:41:50.658846 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 611/1203] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:41:50.666293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc664bc-dfdf-4afd-b5be-9fd27bc491fc None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:50.800657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc664bc-dfdf-4afd-b5be-9fd27bc491fc None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:50.847135 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:50.859026 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71312fc5-c2e1-4e96-89f5-dc7568740c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:50.859743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71312fc5-c2e1-4e96-89f5-dc7568740c24 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:50.860052 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71312fc5-c2e1-4e96-89f5-dc7568740c24 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:50.860327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71312fc5-c2e1-4e96-89f5-dc7568740c24 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:50.861989 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71312fc5-c2e1-4e96-89f5-dc7568740c24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:50.862316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71312fc5-c2e1-4e96-89f5-dc7568740c24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:50.863917 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71312fc5-c2e1-4e96-89f5-dc7568740c24 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:50.877768 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 593/1204] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:41:50.883438 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2379b3-5286-453c-9175-a7647fd8ce0f None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:50.957412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:51.016522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2379b3-5286-453c-9175-a7647fd8ce0f None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:51.058791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210c660-dcbe-4749-90e1-1c2cc21bae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:51.059478 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210c660-dcbe-4749-90e1-1c2cc21bae67 None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:51.059837 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210c660-dcbe-4749-90e1-1c2cc21bae67 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:51.060136 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210c660-dcbe-4749-90e1-1c2cc21bae67 None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:51.060970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c210c660-dcbe-4749-90e1-1c2cc21bae67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:51.062462 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c210c660-dcbe-4749-90e1-1c2cc21bae67 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:51.088277 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 612/1205] 10.222.0.60 () {60 vars in 1132 bytes} [Thu Jul 21 20:41:50 2022] POST /identity/v3/users => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:41:53.305778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab34462-9147-45b0-81ee-e9cbce1cebbe None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:53.306332 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab34462-9147-45b0-81ee-e9cbce1cebbe None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:53.306332 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab34462-9147-45b0-81ee-e9cbce1cebbe None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:53.307915 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 594/1206] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:41:53.313694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd18bc-551c-4e06-b9de-df74bc15caa6 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:53.314736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd18bc-551c-4e06-b9de-df74bc15caa6 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:53.314829 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd18bc-551c-4e06-b9de-df74bc15caa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:53.332887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-13bd18bc-551c-4e06-b9de-df74bc15caa6 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:53.334086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-13bd18bc-551c-4e06-b9de-df74bc15caa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:53.422435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 613/1207] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:41:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:41:53.448276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab34462-9147-45b0-81ee-e9cbce1cebbe None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:53.583424 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab34462-9147-45b0-81ee-e9cbce1cebbe None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:53.636966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f2ff2-643c-4997-982a-b98907c25d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:53.639658 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f2ff2-643c-4997-982a-b98907c25d15 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:53.639658 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f2ff2-643c-4997-982a-b98907c25d15 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:53.639658 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f2ff2-643c-4997-982a-b98907c25d15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:53.651138 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f2ff2-643c-4997-982a-b98907c25d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-db4f2ff2-643c-4997-982a-b98907c25d15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:41:53.660296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:41:53.661778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:41:53.662918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:41:53.662918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:41:53.662918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:41:53.662918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:41:53.662918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:41:53.662918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:41:53.662918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:41:53.662918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:41:53.662918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:41:53.662918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:41:53.662918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:41:53.662918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:41:53.662918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 595/1208] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:41:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:41:53.667829 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd18bc-551c-4e06-b9de-df74bc15caa6 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:53.723573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:53.789848 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd18bc-551c-4e06-b9de-df74bc15caa6 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:53.830720 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7a684-8564-4010-a09e-048a42dfda53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:53.832929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7a684-8564-4010-a09e-048a42dfda53 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:53.832929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7a684-8564-4010-a09e-048a42dfda53 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:53.832929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7a684-8564-4010-a09e-048a42dfda53 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:53.832929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7a684-8564-4010-a09e-048a42dfda53 None admin] RBAC: Adding query filter params (name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:53.833302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7a684-8564-4010-a09e-048a42dfda53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:53.834540 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7a684-8564-4010-a09e-048a42dfda53 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:53.843300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 614/1209] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:41:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:41:53.852773 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f2ff2-643c-4997-982a-b98907c25d15 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:53.912206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:53.970121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f2ff2-643c-4997-982a-b98907c25d15 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:54.011762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e58d2-965d-40f8-b723-f455f98c49d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:54.012568 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e58d2-965d-40f8-b723-f455f98c49d6 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:54.012853 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e58d2-965d-40f8-b723-f455f98c49d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:54.013128 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e58d2-965d-40f8-b723-f455f98c49d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:54.025405 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e58d2-965d-40f8-b723-f455f98c49d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:54.027173 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e58d2-965d-40f8-b723-f455f98c49d6 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:54.047882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-290e58d2-965d-40f8-b723-f455f98c49d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:41:54.047882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:41:54.047882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:41:54.047882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:41:54.047882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:41:54.047882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:41:54.047882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:41:54.047882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:41:54.047882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:41:54.047882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:41:54.047882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:41:54.047882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:41:54.047882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:41:54.047882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:41:54.047882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:41:54.049963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:41:54.051937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:41:54.051937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:41:54.051937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:41:54.051937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:41:54.051937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 596/1210] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:41:54.056541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7a684-8564-4010-a09e-048a42dfda53 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:54.187009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:54.191677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7a684-8564-4010-a09e-048a42dfda53 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:54.233373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b128eae-f888-49a8-ad9f-eb1966c9ee58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:54.234055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b128eae-f888-49a8-ad9f-eb1966c9ee58 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:54.234349 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b128eae-f888-49a8-ad9f-eb1966c9ee58 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:54.234628 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b128eae-f888-49a8-ad9f-eb1966c9ee58 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:54.235362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b128eae-f888-49a8-ad9f-eb1966c9ee58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:54.235675 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b128eae-f888-49a8-ad9f-eb1966c9ee58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:54.237834 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b128eae-f888-49a8-ad9f-eb1966c9ee58 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:54.261485 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 615/1211] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:41:54.261724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e58d2-965d-40f8-b723-f455f98c49d6 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:54.382769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e58d2-965d-40f8-b723-f455f98c49d6 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:54.384400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:54.426912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5452d0-4b0b-47b0-a218-e35e115a0992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:54.427814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5452d0-4b0b-47b0-a218-e35e115a0992 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:54.428104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5452d0-4b0b-47b0-a218-e35e115a0992 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:54.428382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5452d0-4b0b-47b0-a218-e35e115a0992 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:54.437473 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5452d0-4b0b-47b0-a218-e35e115a0992 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:54.439191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5452d0-4b0b-47b0-a218-e35e115a0992 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-af5452d0-4b0b-47b0-a218-e35e115a0992 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:41:54.455888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:41:54.457276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:41:54.458223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:41:54.458223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:41:54.458223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:41:54.458223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:41:54.458223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:41:54.458223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:41:54.458223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:41:54.458223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:41:54.458223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:41:54.458223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:41:54.458223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 597/1212] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:41:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:41:54.462961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b128eae-f888-49a8-ad9f-eb1966c9ee58 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:54.578495 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b128eae-f888-49a8-ad9f-eb1966c9ee58 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:54.617278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4888b-7285-4002-9320-dbb0445c7a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:54.617925 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4888b-7285-4002-9320-dbb0445c7a0b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:54.618206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4888b-7285-4002-9320-dbb0445c7a0b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:54.618529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4888b-7285-4002-9320-dbb0445c7a0b None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:54.619434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4888b-7285-4002-9320-dbb0445c7a0b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:54.619808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4888b-7285-4002-9320-dbb0445c7a0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:54.621715 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4888b-7285-4002-9320-dbb0445c7a0b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:54.646988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 616/1213] 10.222.0.60 () {56 vars in 1143 bytes} [Thu Jul 21 20:41:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:41:54.648203 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5452d0-4b0b-47b0-a218-e35e115a0992 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:54.772632 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5452d0-4b0b-47b0-a218-e35e115a0992 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:54.820328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9aa0b1-164c-4581-8654-84a688da363c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:54.821045 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9aa0b1-164c-4581-8654-84a688da363c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:54.822122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9aa0b1-164c-4581-8654-84a688da363c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:54.822556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9aa0b1-164c-4581-8654-84a688da363c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:54.859098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:54.912232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9aa0b1-164c-4581-8654-84a688da363c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:54.913854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9aa0b1-164c-4581-8654-84a688da363c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:55.054255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 598/1214] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:41:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:41:55.060823 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4888b-7285-4002-9320-dbb0445c7a0b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:55.085013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:55.188891 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4888b-7285-4002-9320-dbb0445c7a0b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:55.231402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980d805-e9eb-406e-ac06-550617874490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:55.232302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980d805-e9eb-406e-ac06-550617874490 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:55.232613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980d805-e9eb-406e-ac06-550617874490 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:55.232933 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980d805-e9eb-406e-ac06-550617874490 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:55.248605 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980d805-e9eb-406e-ac06-550617874490 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:55.250141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980d805-e9eb-406e-ac06-550617874490 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:55.274114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-4980d805-e9eb-406e-ac06-550617874490 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:41:55.274114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:41:55.274114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:41:55.274114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:41:55.274114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:41:55.274114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:41:55.274114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:41:55.274114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:41:55.274114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:41:55.274114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:41:55.274114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:41:55.274114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:41:55.274114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:41:55.274114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:41:55.274114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:41:55.275679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:41:55.276947 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:41:55.276947 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:41:55.276947 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:41:55.276947 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:41:55.276947 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 617/1215] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:41:55.281319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9aa0b1-164c-4581-8654-84a688da363c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:55.360274 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:55.416771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9aa0b1-164c-4581-8654-84a688da363c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:55.455968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e0244-f340-4c58-80e7-9729266264da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:55.456659 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e0244-f340-4c58-80e7-9729266264da None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:55.456941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e0244-f340-4c58-80e7-9729266264da None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:55.457215 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e0244-f340-4c58-80e7-9729266264da None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:55.457946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e0244-f340-4c58-80e7-9729266264da None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:55.458251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e0244-f340-4c58-80e7-9729266264da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:55.459663 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e0244-f340-4c58-80e7-9729266264da None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:55.473782 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 599/1216] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:41:55.478222 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980d805-e9eb-406e-ac06-550617874490 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:55.580345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:55.619827 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980d805-e9eb-406e-ac06-550617874490 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:55.666850 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550d301-47ff-4cda-aa11-54088dcd907d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:55.667826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550d301-47ff-4cda-aa11-54088dcd907d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:55.668239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550d301-47ff-4cda-aa11-54088dcd907d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:55.668531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550d301-47ff-4cda-aa11-54088dcd907d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:55.677132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550d301-47ff-4cda-aa11-54088dcd907d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:55.678829 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550d301-47ff-4cda-aa11-54088dcd907d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:55.696550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-6550d301-47ff-4cda-aa11-54088dcd907d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:41:55.696550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:41:55.696550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:41:55.696550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:41:55.696550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:41:55.696550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:41:55.696550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:41:55.696550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:41:55.696550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:41:55.696550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:41:55.696550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:41:55.696550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:41:55.696550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:41:55.696550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:41:55.696550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:41:55.697688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:41:55.698592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:41:55.698592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:41:55.698592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:41:55.698592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:41:55.698592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:41:55.698592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:41:55.698592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:41:55.698592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:41:55.698592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 618/1217] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:41:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:41:55.704208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e0244-f340-4c58-80e7-9729266264da None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:55.833268 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e0244-f340-4c58-80e7-9729266264da None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:55.864703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:55.879247 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ed671-b4a9-46ec-a7f0-12cb6fb188ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:55.879878 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ed671-b4a9-46ec-a7f0-12cb6fb188ea None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:55.880238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ed671-b4a9-46ec-a7f0-12cb6fb188ea None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:55.880534 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ed671-b4a9-46ec-a7f0-12cb6fb188ea None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:55.881383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ed671-b4a9-46ec-a7f0-12cb6fb188ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:55.881689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ed671-b4a9-46ec-a7f0-12cb6fb188ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:55.883117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ed671-b4a9-46ec-a7f0-12cb6fb188ea None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:55.898841 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 600/1218] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:41:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:41:55.906086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550d301-47ff-4cda-aa11-54088dcd907d None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:56.024674 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550d301-47ff-4cda-aa11-54088dcd907d None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:56.058966 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:56.068095 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44110f1e-a342-4e58-ab66-a7665f514fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:56.069719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44110f1e-a342-4e58-ab66-a7665f514fcd None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:56.069719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44110f1e-a342-4e58-ab66-a7665f514fcd None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:56.069719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44110f1e-a342-4e58-ab66-a7665f514fcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:56.070394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44110f1e-a342-4e58-ab66-a7665f514fcd None admin] RBAC: Adding query filter params (role.id=05f3681d0a644607934e281f75ecd675, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=f4786051321f4a6f80a6cd23eacb09bc) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:56.071003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44110f1e-a342-4e58-ab66-a7665f514fcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:56.072546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44110f1e-a342-4e58-ab66-a7665f514fcd None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:56.081016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 619/1219] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:41:55 2022] GET /identity/v3/role_assignments?user.id=f4786051321f4a6f80a6cd23eacb09bc&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=05f3681d0a644607934e281f75ecd675 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:41:58.223122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578096b-ddf9-4a23-b993-4a280cfdd784 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:58.223669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578096b-ddf9-4a23-b993-4a280cfdd784 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:58.223669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578096b-ddf9-4a23-b993-4a280cfdd784 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:58.225091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 601/1220] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:41:58.231144 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d43f47-714e-4f42-aed8-9e9109c77818 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:58.231627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d43f47-714e-4f42-aed8-9e9109c77818 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:58.231821 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d43f47-714e-4f42-aed8-9e9109c77818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:58.248693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-37d43f47-714e-4f42-aed8-9e9109c77818 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:41:58.249890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-37d43f47-714e-4f42-aed8-9e9109c77818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:41:58.327770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 620/1221] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:41:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:41:58.353062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578096b-ddf9-4a23-b993-4a280cfdd784 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:58.466092 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578096b-ddf9-4a23-b993-4a280cfdd784 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:58.504006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be8111-1449-43c1-808f-9b9477586620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:58.504733 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be8111-1449-43c1-808f-9b9477586620 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:58.505017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be8111-1449-43c1-808f-9b9477586620 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:58.505290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be8111-1449-43c1-808f-9b9477586620 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:58.517001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be8111-1449-43c1-808f-9b9477586620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-e9be8111-1449-43c1-808f-9b9477586620 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:41:58.525399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:41:58.526409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:41:58.527233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:41:58.527233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:41:58.527233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:41:58.527233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:41:58.527233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:41:58.527233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:41:58.527233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:41:58.527233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:41:58.527233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:41:58.527233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:41:58.527233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:41:58.527233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:41:58.527233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 602/1222] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:41:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:41:58.535234 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d43f47-714e-4f42-aed8-9e9109c77818 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:58.660459 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d43f47-714e-4f42-aed8-9e9109c77818 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:58.688229 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:58.700895 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c905c5-1397-40f4-bcd2-23574206bec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:58.701598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c905c5-1397-40f4-bcd2-23574206bec1 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:58.701857 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c905c5-1397-40f4-bcd2-23574206bec1 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:58.702153 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c905c5-1397-40f4-bcd2-23574206bec1 None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:58.702927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c905c5-1397-40f4-bcd2-23574206bec1 None admin] RBAC: Adding query filter params (name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:58.703235 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c905c5-1397-40f4-bcd2-23574206bec1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:58.705288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c905c5-1397-40f4-bcd2-23574206bec1 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:58.714138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 621/1223] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:41:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:41:58.720544 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be8111-1449-43c1-808f-9b9477586620 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:58.770994 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:58.841804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be8111-1449-43c1-808f-9b9477586620 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:58.880382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779f797-8e8f-422a-ae7e-353488df2a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:58.881310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779f797-8e8f-422a-ae7e-353488df2a02 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:58.881595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779f797-8e8f-422a-ae7e-353488df2a02 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:58.881920 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779f797-8e8f-422a-ae7e-353488df2a02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:58.894275 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779f797-8e8f-422a-ae7e-353488df2a02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:58.895844 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779f797-8e8f-422a-ae7e-353488df2a02 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:58.914487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-e779f797-8e8f-422a-ae7e-353488df2a02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:41:58.914487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:41:58.914487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:41:58.914487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:41:58.914487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:41:58.914487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:41:58.914487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:41:58.914487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:41:58.914487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:41:58.914487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:41:58.914487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:41:58.914487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:41:58.914487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:41:58.914487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:41:58.914487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:41:58.922868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:41:58.925141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:41:58.925141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:41:58.925141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:41:58.925141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:41:58.925141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 603/1224] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:41:58.925721 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c905c5-1397-40f4-bcd2-23574206bec1 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:59.046863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c905c5-1397-40f4-bcd2-23574206bec1 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:59.082554 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d6c1c-0854-466a-947d-dc43a8df9a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:59.083340 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d6c1c-0854-466a-947d-dc43a8df9a6e None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:59.083632 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d6c1c-0854-466a-947d-dc43a8df9a6e None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:59.084048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d6c1c-0854-466a-947d-dc43a8df9a6e None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:59.084736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d6c1c-0854-466a-947d-dc43a8df9a6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:59.085035 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d6c1c-0854-466a-947d-dc43a8df9a6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:59.086631 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d6c1c-0854-466a-947d-dc43a8df9a6e None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:59.103802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 622/1225] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:41:59.110314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779f797-8e8f-422a-ae7e-353488df2a02 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:59.218614 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:59.226687 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779f797-8e8f-422a-ae7e-353488df2a02 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:59.265262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e79d8e-80e7-470a-999b-e02c2070140d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:59.266062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e79d8e-80e7-470a-999b-e02c2070140d None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:59.266340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e79d8e-80e7-470a-999b-e02c2070140d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:59.266609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e79d8e-80e7-470a-999b-e02c2070140d None admin] PATH_INFO: `/v3/users/placement` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:59.274171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e79d8e-80e7-470a-999b-e02c2070140d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:59.275602 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e79d8e-80e7-470a-999b-e02c2070140d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-17e79d8e-80e7-470a-999b-e02c2070140d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:41:59.290555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:41:59.291914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:41:59.293397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:41:59.293397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:41:59.293397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:41:59.293397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:41:59.293397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:41:59.293397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:41:59.293397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:41:59.293397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:41:59.293397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:41:59.293397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:41:59.293397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 604/1226] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:41:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:41:59.298882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d6c1c-0854-466a-947d-dc43a8df9a6e None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:59.357633 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:59.416584 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d6c1c-0854-466a-947d-dc43a8df9a6e None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:59.453365 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938569b6-db6c-4b02-9085-e9cd433b4f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:59.454029 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938569b6-db6c-4b02-9085-e9cd433b4f9d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:59.454307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938569b6-db6c-4b02-9085-e9cd433b4f9d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:59.454608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938569b6-db6c-4b02-9085-e9cd433b4f9d None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:59.455434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938569b6-db6c-4b02-9085-e9cd433b4f9d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:59.455768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938569b6-db6c-4b02-9085-e9cd433b4f9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:59.457193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938569b6-db6c-4b02-9085-e9cd433b4f9d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:59.475110 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 623/1227] 10.222.0.60 () {56 vars in 1143 bytes} [Thu Jul 21 20:41:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:41:59.481129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e79d8e-80e7-470a-999b-e02c2070140d None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:59.594763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e79d8e-80e7-470a-999b-e02c2070140d None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:59.641569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc6393-d4c1-46fd-9964-d22b851b844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:59.642660 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc6393-d4c1-46fd-9964-d22b851b844e None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:59.642946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc6393-d4c1-46fd-9964-d22b851b844e None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:59.643201 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc6393-d4c1-46fd-9964-d22b851b844e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:59.654307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc6393-d4c1-46fd-9964-d22b851b844e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:59.655826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc6393-d4c1-46fd-9964-d22b851b844e None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:59.674502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-74dc6393-d4c1-46fd-9964-d22b851b844e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:41:59.674502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:41:59.674502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:41:59.674502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:41:59.674502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:41:59.674502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:41:59.674502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:41:59.674502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:41:59.674502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:41:59.674502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:41:59.674502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:41:59.674502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:41:59.674502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:41:59.674502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:41:59.674502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:41:59.675587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:41:59.676505 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:41:59.676505 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:41:59.676505 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:41:59.676505 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:41:59.676505 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 605/1228] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:41:59.681426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938569b6-db6c-4b02-9085-e9cd433b4f9d None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:59.796669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938569b6-db6c-4b02-9085-e9cd433b4f9d None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:41:59.799404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:59.832447 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43f342-799a-4d84-b3c7-458e4158eaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:41:59.833123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43f342-799a-4d84-b3c7-458e4158eaaa None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:41:59.833406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43f342-799a-4d84-b3c7-458e4158eaaa None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:41:59.835174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43f342-799a-4d84-b3c7-458e4158eaaa None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:41:59.835174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43f342-799a-4d84-b3c7-458e4158eaaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:41:59.835174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43f342-799a-4d84-b3c7-458e4158eaaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:41:59.836103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43f342-799a-4d84-b3c7-458e4158eaaa None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:41:59.849222 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 624/1229] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:41:59.855222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc6393-d4c1-46fd-9964-d22b851b844e None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:41:59.866437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:41:59.963439 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc6393-d4c1-46fd-9964-d22b851b844e None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:00.006413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e1bc1-bc68-4c8f-9347-ba01f97b15e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:00.007214 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e1bc1-bc68-4c8f-9347-ba01f97b15e7 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:00.007497 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e1bc1-bc68-4c8f-9347-ba01f97b15e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:00.007800 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e1bc1-bc68-4c8f-9347-ba01f97b15e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:00.015067 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e1bc1-bc68-4c8f-9347-ba01f97b15e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:00.016606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e1bc1-bc68-4c8f-9347-ba01f97b15e7 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:00.036806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-dd7e1bc1-bc68-4c8f-9347-ba01f97b15e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:42:00.036806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:00.036806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:00.036806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:00.036806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:00.036806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:00.036806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:00.036806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:00.036806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:00.036806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:00.036806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:00.036806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:00.036806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:42:00.036806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:42:00.036806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:00.037798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:00.038668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:42:00.038668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:42:00.038668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:42:00.038668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:42:00.038668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:42:00.038668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:42:00.038668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:42:00.038668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:00.038668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 606/1230] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:41:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:42:00.044484 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43f342-799a-4d84-b3c7-458e4158eaaa None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:00.166283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43f342-799a-4d84-b3c7-458e4158eaaa None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:00.203226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70a8b3-4fc9-409a-b57a-53cc2db2e765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:00.203931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70a8b3-4fc9-409a-b57a-53cc2db2e765 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:00.204216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70a8b3-4fc9-409a-b57a-53cc2db2e765 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:00.204610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70a8b3-4fc9-409a-b57a-53cc2db2e765 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:00.205876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70a8b3-4fc9-409a-b57a-53cc2db2e765 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:00.206350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70a8b3-4fc9-409a-b57a-53cc2db2e765 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:00.207942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70a8b3-4fc9-409a-b57a-53cc2db2e765 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:00.223475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 625/1231] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:42:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:42:00.229910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e1bc1-bc68-4c8f-9347-ba01f97b15e7 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:00.316569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:00.366954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e1bc1-bc68-4c8f-9347-ba01f97b15e7 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:00.413028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a849532-f43e-47e5-80ff-f5a9367a65c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:00.413959 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a849532-f43e-47e5-80ff-f5a9367a65c2 None admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:00.414269 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a849532-f43e-47e5-80ff-f5a9367a65c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:00.414552 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a849532-f43e-47e5-80ff-f5a9367a65c2 None admin] PATH_INFO: `/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/f4786051321f4a6f80a6cd23eacb09bc/roles/05f3681d0a644607934e281f75ecd675` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:00.432356 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a849532-f43e-47e5-80ff-f5a9367a65c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc61fe15894891a9054911321f4ac3, user_id=f4786051321f4a6f80a6cd23eacb09bc, role_id=05f3681d0a644607934e281f75ecd675)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:00.433865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a849532-f43e-47e5-80ff-f5a9367a65c2 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:00.448442 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 607/1232] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:42:00 2022] PUT /identity/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/f4786051321f4a6f80a6cd23eacb09bc/roles/05f3681d0a644607934e281f75ecd675 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:42:02.640635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9703fe-e0b2-46f1-bb4f-a80c28308724 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:02.641178 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9703fe-e0b2-46f1-bb4f-a80c28308724 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:02.641178 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9703fe-e0b2-46f1-bb4f-a80c28308724 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:02.643231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 626/1233] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:42:02.649209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21217c4-4545-457a-91d9-73f44e29c02f None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:02.649686 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21217c4-4545-457a-91d9-73f44e29c02f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:02.649791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21217c4-4545-457a-91d9-73f44e29c02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:02.668678 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d21217c4-4545-457a-91d9-73f44e29c02f None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:02.670084 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d21217c4-4545-457a-91d9-73f44e29c02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:02.752336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 608/1234] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:42:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:42:02.778450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9703fe-e0b2-46f1-bb4f-a80c28308724 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:02.807565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:02.903553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9703fe-e0b2-46f1-bb4f-a80c28308724 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:02.940291 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b32921-12b0-443c-aaf3-ef06b9e18258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:02.941011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b32921-12b0-443c-aaf3-ef06b9e18258 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:02.941293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b32921-12b0-443c-aaf3-ef06b9e18258 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:02.941710 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b32921-12b0-443c-aaf3-ef06b9e18258 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:02.952401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b32921-12b0-443c-aaf3-ef06b9e18258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-a9b32921-12b0-443c-aaf3-ef06b9e18258 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:42:02.960696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:02.961897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:02.962980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:02.962980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:02.962980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:02.962980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:02.962980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:42:02.962980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:42:02.962980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:42:02.962980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:42:02.962980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:42:02.962980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:42:02.962980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 20:42:02.962980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:42:02.962980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 627/1235] 10.222.0.60 () {56 vars in 1102 bytes} [Thu Jul 21 20:42:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:02.968113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21217c4-4545-457a-91d9-73f44e29c02f None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:03.023498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:03.087525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21217c4-4545-457a-91d9-73f44e29c02f None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:03.132827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9f368-5770-4cd5-9a51-6f71b80e5951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:03.133517 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9f368-5770-4cd5-9a51-6f71b80e5951 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:03.133800 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9f368-5770-4cd5-9a51-6f71b80e5951 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:03.134075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9f368-5770-4cd5-9a51-6f71b80e5951 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:03.134832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9f368-5770-4cd5-9a51-6f71b80e5951 None admin] RBAC: Adding query filter params (name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:03.135136 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9f368-5770-4cd5-9a51-6f71b80e5951 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:03.136555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9f368-5770-4cd5-9a51-6f71b80e5951 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:03.147880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 609/1236] 10.222.0.60 () {56 vars in 1103 bytes} [Thu Jul 21 20:42:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:03.154123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b32921-12b0-443c-aaf3-ef06b9e18258 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:03.249272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:03.266611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b32921-12b0-443c-aaf3-ef06b9e18258 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:03.307528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55971552-7c67-46f0-a2da-12beadd745cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:03.308735 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55971552-7c67-46f0-a2da-12beadd745cd None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:03.309242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55971552-7c67-46f0-a2da-12beadd745cd None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:03.309736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55971552-7c67-46f0-a2da-12beadd745cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:03.327907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55971552-7c67-46f0-a2da-12beadd745cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:03.331187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55971552-7c67-46f0-a2da-12beadd745cd None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:03.352762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-55971552-7c67-46f0-a2da-12beadd745cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:03.352762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:03.352762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:03.352762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:03.352762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:03.352762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:03.352762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:03.352762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:03.352762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:03.352762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:03.352762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:03.352762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:03.352762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:42:03.352762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:42:03.352762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:03.353905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:03.360049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:42:03.360049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:42:03.360049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:03.360049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:42:03.360049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 628/1237] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:03.361854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9f368-5770-4cd5-9a51-6f71b80e5951 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:03.483738 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9f368-5770-4cd5-9a51-6f71b80e5951 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:03.493959 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:03.521403 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4a500-1617-4678-9e8a-ac6e4ae54a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:03.521990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4a500-1617-4678-9e8a-ac6e4ae54a0f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:03.522272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4a500-1617-4678-9e8a-ac6e4ae54a0f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:03.522665 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4a500-1617-4678-9e8a-ac6e4ae54a0f None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:03.523398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4a500-1617-4678-9e8a-ac6e4ae54a0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:03.523703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4a500-1617-4678-9e8a-ac6e4ae54a0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:03.525092 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4a500-1617-4678-9e8a-ac6e4ae54a0f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:03.538070 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 610/1238] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:03.544517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55971552-7c67-46f0-a2da-12beadd745cd None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:03.663787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55971552-7c67-46f0-a2da-12beadd745cd None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:03.703729 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af019b42-e83e-4a34-97d1-c5ed86a2cc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:03.704562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af019b42-e83e-4a34-97d1-c5ed86a2cc16 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:03.704851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af019b42-e83e-4a34-97d1-c5ed86a2cc16 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:03.705140 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af019b42-e83e-4a34-97d1-c5ed86a2cc16 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:03.708826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:03.714212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af019b42-e83e-4a34-97d1-c5ed86a2cc16 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:03.715684 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af019b42-e83e-4a34-97d1-c5ed86a2cc16 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-af019b42-e83e-4a34-97d1-c5ed86a2cc16 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:42:03.730313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:03.731893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:03.733685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:03.733685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:03.733685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:42:03.733685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:42:03.733685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:42:03.733685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:42:03.733685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:42:03.733685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:42:03.733685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:42:03.733685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:42:03.737309 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 629/1239] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:42:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:03.738782 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4a500-1617-4678-9e8a-ac6e4ae54a0f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:03.858947 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4a500-1617-4678-9e8a-ac6e4ae54a0f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:03.899103 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8254f-6203-49ad-bfdf-44ac579d7a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:03.899822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8254f-6203-49ad-bfdf-44ac579d7a39 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:03.900112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8254f-6203-49ad-bfdf-44ac579d7a39 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:03.900393 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8254f-6203-49ad-bfdf-44ac579d7a39 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:03.901236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8254f-6203-49ad-bfdf-44ac579d7a39 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:03.901633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8254f-6203-49ad-bfdf-44ac579d7a39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:03.903059 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8254f-6203-49ad-bfdf-44ac579d7a39 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:03.935602 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 611/1240] 10.222.0.60 () {56 vars in 1143 bytes} [Thu Jul 21 20:42:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:03.937046 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af019b42-e83e-4a34-97d1-c5ed86a2cc16 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:04.048467 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af019b42-e83e-4a34-97d1-c5ed86a2cc16 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:04.084542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfe945-b391-41bb-ae13-6b01d3c7a706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:04.085262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfe945-b391-41bb-ae13-6b01d3c7a706 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:04.085569 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfe945-b391-41bb-ae13-6b01d3c7a706 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:04.085909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfe945-b391-41bb-ae13-6b01d3c7a706 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:04.165289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:04.172874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfe945-b391-41bb-ae13-6b01d3c7a706 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:04.175097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfe945-b391-41bb-ae13-6b01d3c7a706 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:04.304018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 630/1241] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:42:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:42:04.305531 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8254f-6203-49ad-bfdf-44ac579d7a39 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:04.334823 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:04.415518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8254f-6203-49ad-bfdf-44ac579d7a39 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:04.458756 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374237f-2969-4a99-9181-ea2bae4ddafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:04.459566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374237f-2969-4a99-9181-ea2bae4ddafb None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:04.459881 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374237f-2969-4a99-9181-ea2bae4ddafb None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:04.460161 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374237f-2969-4a99-9181-ea2bae4ddafb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:04.470877 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374237f-2969-4a99-9181-ea2bae4ddafb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:04.472522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374237f-2969-4a99-9181-ea2bae4ddafb None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:04.491595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-d374237f-2969-4a99-9181-ea2bae4ddafb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:04.491595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:04.491595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:04.491595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:04.491595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:04.491595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:04.491595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:04.491595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:04.491595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:04.491595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:04.491595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:04.491595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:04.491595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:42:04.491595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:42:04.491595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:04.492757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:04.493825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:42:04.493825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:42:04.493825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:04.493825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:04.493825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 612/1242] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:04.499018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfe945-b391-41bb-ae13-6b01d3c7a706 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:04.603975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:04.610560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfe945-b391-41bb-ae13-6b01d3c7a706 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:04.650952 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd2b62-345d-46ae-9201-f775846c0d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:04.651798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd2b62-345d-46ae-9201-f775846c0d4d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:04.654400 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd2b62-345d-46ae-9201-f775846c0d4d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:04.654737 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd2b62-345d-46ae-9201-f775846c0d4d None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:04.655523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd2b62-345d-46ae-9201-f775846c0d4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:04.655864 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd2b62-345d-46ae-9201-f775846c0d4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:04.658970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd2b62-345d-46ae-9201-f775846c0d4d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:04.678035 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 631/1243] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:04.679362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374237f-2969-4a99-9181-ea2bae4ddafb None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:04.773693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:04.806440 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374237f-2969-4a99-9181-ea2bae4ddafb None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:04.842061 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f2702-acb5-4dcd-bd27-7b5fa68ebc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:04.843223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f2702-acb5-4dcd-bd27-7b5fa68ebc13 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:04.843695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f2702-acb5-4dcd-bd27-7b5fa68ebc13 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:04.844010 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f2702-acb5-4dcd-bd27-7b5fa68ebc13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:04.851727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f2702-acb5-4dcd-bd27-7b5fa68ebc13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:04.853363 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f2702-acb5-4dcd-bd27-7b5fa68ebc13 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:04.868621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-f48f2702-acb5-4dcd-bd27-7b5fa68ebc13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:42:04.868621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:04.868621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:04.868621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:04.868621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:04.868621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:04.868621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:04.868621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:04.868621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:04.868621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:04.868621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:04.868621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:04.868621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:42:04.868621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:42:04.868621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:04.869581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:04.870539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:42:04.870539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:42:04.870539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:42:04.870539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:42:04.870539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:42:04.870539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:42:04.870539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:42:04.870539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:04.870539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 613/1244] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:42:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:04.875845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd2b62-345d-46ae-9201-f775846c0d4d None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:04.988558 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd2b62-345d-46ae-9201-f775846c0d4d None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:05.029658 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff53ddc-cb79-4131-b500-ecf5b896a8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:05.030417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff53ddc-cb79-4131-b500-ecf5b896a8c8 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:05.030710 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff53ddc-cb79-4131-b500-ecf5b896a8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:05.030913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff53ddc-cb79-4131-b500-ecf5b896a8c8 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:05.031702 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff53ddc-cb79-4131-b500-ecf5b896a8c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:05.032054 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff53ddc-cb79-4131-b500-ecf5b896a8c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:05.034225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff53ddc-cb79-4131-b500-ecf5b896a8c8 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:05.047959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 632/1245] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:42:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:05.054797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f2702-acb5-4dcd-bd27-7b5fa68ebc13 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:05.185325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f2702-acb5-4dcd-bd27-7b5fa68ebc13 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:05.226451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af7422-ea48-485a-94f9-eb18feacecfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:05.227107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af7422-ea48-485a-94f9-eb18feacecfb None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:05.227387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af7422-ea48-485a-94f9-eb18feacecfb None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:05.227656 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af7422-ea48-485a-94f9-eb18feacecfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:05.228554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af7422-ea48-485a-94f9-eb18feacecfb None admin] RBAC: Adding query filter params (role.id=05f3681d0a644607934e281f75ecd675, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=f4786051321f4a6f80a6cd23eacb09bc) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:05.228831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af7422-ea48-485a-94f9-eb18feacecfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:05.230342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af7422-ea48-485a-94f9-eb18feacecfb None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:05.238248 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 614/1246] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:42:05 2022] GET /identity/v3/role_assignments?user.id=f4786051321f4a6f80a6cd23eacb09bc&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=05f3681d0a644607934e281f75ecd675 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:07.777109 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21918f00-8eb9-444a-8201-a2a952a276af None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:07.777625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21918f00-8eb9-444a-8201-a2a952a276af None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:07.777625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21918f00-8eb9-444a-8201-a2a952a276af None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:07.778938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 633/1247] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:42:07.785715 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff1a52-e8b5-4e31-a463-495e29605674 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:07.786198 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff1a52-e8b5-4e31-a463-495e29605674 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:07.786592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff1a52-e8b5-4e31-a463-495e29605674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:07.792492 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:07.805872 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-9fff1a52-e8b5-4e31-a463-495e29605674 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:07.807093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-9fff1a52-e8b5-4e31-a463-495e29605674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:07.893227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 615/1248] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:42:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:42:07.919219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21918f00-8eb9-444a-8201-a2a952a276af None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:07.942051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:08.039889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21918f00-8eb9-444a-8201-a2a952a276af None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:08.082266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7683d24-745b-4e19-b7bc-ffb5b36e6781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:08.083538 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7683d24-745b-4e19-b7bc-ffb5b36e6781 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:08.084051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7683d24-745b-4e19-b7bc-ffb5b36e6781 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:08.084546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7683d24-745b-4e19-b7bc-ffb5b36e6781 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:08.097902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7683d24-745b-4e19-b7bc-ffb5b36e6781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-f7683d24-745b-4e19-b7bc-ffb5b36e6781 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:42:08.112166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:08.114419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:08.115329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:08.115329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:08.115329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:08.115329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:08.115329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:42:08.115329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:42:08.115329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:42:08.115329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:42:08.115329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:42:08.115329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:42:08.115329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:42:08.115329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:42:08.115329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 634/1249] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:42:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:08.121117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff1a52-e8b5-4e31-a463-495e29605674 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:08.267537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff1a52-e8b5-4e31-a463-495e29605674 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:08.301632 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:08.307915 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe3d07-a823-4045-befe-245fe7925c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:08.308883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe3d07-a823-4045-befe-245fe7925c67 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:08.309320 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe3d07-a823-4045-befe-245fe7925c67 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:08.309666 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe3d07-a823-4045-befe-245fe7925c67 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:08.310643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe3d07-a823-4045-befe-245fe7925c67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:08.311051 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe3d07-a823-4045-befe-245fe7925c67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:08.312812 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe3d07-a823-4045-befe-245fe7925c67 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:08.322931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 616/1250] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:42:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:08.329308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7683d24-745b-4e19-b7bc-ffb5b36e6781 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:08.430410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:08.461820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7683d24-745b-4e19-b7bc-ffb5b36e6781 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:08.503261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbbb7f-02ae-48f3-a2c7-2c6929e8d978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:08.504097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbbb7f-02ae-48f3-a2c7-2c6929e8d978 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:08.504389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbbb7f-02ae-48f3-a2c7-2c6929e8d978 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:08.504666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbbb7f-02ae-48f3-a2c7-2c6929e8d978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:08.516498 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbbb7f-02ae-48f3-a2c7-2c6929e8d978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:08.518127 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbbb7f-02ae-48f3-a2c7-2c6929e8d978 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:08.537547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-14bbbb7f-02ae-48f3-a2c7-2c6929e8d978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:08.537547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:08.537547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:08.537547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:08.537547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:08.537547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:08.537547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:08.537547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:08.537547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:08.537547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:08.537547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:08.537547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:08.537547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:42:08.537547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:42:08.537547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:08.538581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:08.539773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:42:08.539773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:42:08.539773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:08.539773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:42:08.545479 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 635/1251] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:08.545614 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe3d07-a823-4045-befe-245fe7925c67 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:08.669377 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe3d07-a823-4045-befe-245fe7925c67 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:08.708698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28de0c5-5ef8-40fd-b9db-4b08e5c8cf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:08.709369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28de0c5-5ef8-40fd-b9db-4b08e5c8cf2c None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:08.709653 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28de0c5-5ef8-40fd-b9db-4b08e5c8cf2c None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:08.709932 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28de0c5-5ef8-40fd-b9db-4b08e5c8cf2c None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:08.710660 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28de0c5-5ef8-40fd-b9db-4b08e5c8cf2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:08.710979 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28de0c5-5ef8-40fd-b9db-4b08e5c8cf2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:08.712369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28de0c5-5ef8-40fd-b9db-4b08e5c8cf2c None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:08.725504 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 617/1252] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:08.731425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbbb7f-02ae-48f3-a2c7-2c6929e8d978 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:08.852100 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbbb7f-02ae-48f3-a2c7-2c6929e8d978 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:08.885420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:08.896323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db778f6a-c6bc-42be-94c3-561b5c028802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:08.897242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db778f6a-c6bc-42be-94c3-561b5c028802 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:08.897590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db778f6a-c6bc-42be-94c3-561b5c028802 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:08.897835 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db778f6a-c6bc-42be-94c3-561b5c028802 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:08.906833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db778f6a-c6bc-42be-94c3-561b5c028802 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:08.908510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db778f6a-c6bc-42be-94c3-561b5c028802 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-db778f6a-c6bc-42be-94c3-561b5c028802 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:42:08.924385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:08.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:08.928517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:08.928517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:08.928517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:42:08.928517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:42:08.928517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:42:08.928517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:42:08.928517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:42:08.928517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:42:08.928517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:42:08.928517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:42:08.928517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 636/1253] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:42:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:08.932325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28de0c5-5ef8-40fd-b9db-4b08e5c8cf2c None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:09.000522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:09.056808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28de0c5-5ef8-40fd-b9db-4b08e5c8cf2c None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:09.098182 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be351e97-623b-4c93-a094-d91355c112a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:09.098964 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be351e97-623b-4c93-a094-d91355c112a7 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:09.099252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be351e97-623b-4c93-a094-d91355c112a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:09.099596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be351e97-623b-4c93-a094-d91355c112a7 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:09.101297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be351e97-623b-4c93-a094-d91355c112a7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:09.101979 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be351e97-623b-4c93-a094-d91355c112a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:09.105092 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be351e97-623b-4c93-a094-d91355c112a7 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:09.135169 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 618/1254] 10.222.0.60 () {56 vars in 1143 bytes} [Thu Jul 21 20:42:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:42:09.142444 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db778f6a-c6bc-42be-94c3-561b5c028802 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:09.269427 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db778f6a-c6bc-42be-94c3-561b5c028802 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:09.306634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f1980-8e11-4d05-b3ac-5a0d44f54e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:09.307414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f1980-8e11-4d05-b3ac-5a0d44f54e0d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:09.307768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f1980-8e11-4d05-b3ac-5a0d44f54e0d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:09.308182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f1980-8e11-4d05-b3ac-5a0d44f54e0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:09.375106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:09.394359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f1980-8e11-4d05-b3ac-5a0d44f54e0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:09.396072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f1980-8e11-4d05-b3ac-5a0d44f54e0d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:09.542351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 637/1255] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:42:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:42:09.543160 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be351e97-623b-4c93-a094-d91355c112a7 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:09.669069 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be351e97-623b-4c93-a094-d91355c112a7 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:09.685413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:09.713663 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82681539-59e3-4a00-a5a6-813f54548d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:09.715125 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82681539-59e3-4a00-a5a6-813f54548d78 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:09.715790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82681539-59e3-4a00-a5a6-813f54548d78 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:09.716404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82681539-59e3-4a00-a5a6-813f54548d78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:09.728623 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82681539-59e3-4a00-a5a6-813f54548d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:09.730061 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82681539-59e3-4a00-a5a6-813f54548d78 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:09.749996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-82681539-59e3-4a00-a5a6-813f54548d78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:09.749996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:09.749996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:09.749996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:09.749996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:09.749996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:09.749996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:09.749996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:09.749996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:09.749996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:09.749996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:09.749996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:09.749996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:42:09.749996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:42:09.749996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:09.754097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:09.755297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:42:09.755297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:42:09.755297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:09.755297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:09.755297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 619/1256] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:09.757769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f1980-8e11-4d05-b3ac-5a0d44f54e0d None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:09.855893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:09.869010 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f1980-8e11-4d05-b3ac-5a0d44f54e0d None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:09.906674 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5455b8-03fc-4180-9c45-a5d59e4623e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:09.907334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5455b8-03fc-4180-9c45-a5d59e4623e5 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:09.907701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5455b8-03fc-4180-9c45-a5d59e4623e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:09.908022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5455b8-03fc-4180-9c45-a5d59e4623e5 None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:09.908767 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5455b8-03fc-4180-9c45-a5d59e4623e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:09.909129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5455b8-03fc-4180-9c45-a5d59e4623e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:09.910824 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5455b8-03fc-4180-9c45-a5d59e4623e5 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:09.926864 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 638/1257] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:09.933432 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82681539-59e3-4a00-a5a6-813f54548d78 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:10.050497 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82681539-59e3-4a00-a5a6-813f54548d78 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:10.089691 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933933a5-4285-4791-885f-30f55ba39927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:10.090585 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933933a5-4285-4791-885f-30f55ba39927 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:10.090889 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933933a5-4285-4791-885f-30f55ba39927 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:10.091211 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933933a5-4285-4791-885f-30f55ba39927 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:10.099446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933933a5-4285-4791-885f-30f55ba39927 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:10.101244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933933a5-4285-4791-885f-30f55ba39927 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:10.118020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-933933a5-4285-4791-885f-30f55ba39927 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:42:10.118020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:10.118020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:10.118020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:10.118020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:10.118020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:10.118020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:10.118020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:10.118020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:10.118020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:10.118020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:10.118020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:10.118020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:42:10.118020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:42:10.118020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:10.119236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:10.120183 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:42:10.120183 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:42:10.120183 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:42:10.120183 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:42:10.120183 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:42:10.120183 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:42:10.120183 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:42:10.120183 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:10.120183 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 620/1258] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:10.125659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5455b8-03fc-4180-9c45-a5d59e4623e5 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:10.242336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5455b8-03fc-4180-9c45-a5d59e4623e5 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:10.277945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d617cb4-6f51-42e9-bd09-4f6a0c72a742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:10.278602 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d617cb4-6f51-42e9-bd09-4f6a0c72a742 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:10.278883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d617cb4-6f51-42e9-bd09-4f6a0c72a742 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:10.279177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d617cb4-6f51-42e9-bd09-4f6a0c72a742 None admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:10.279970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d617cb4-6f51-42e9-bd09-4f6a0c72a742 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:10.280294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d617cb4-6f51-42e9-bd09-4f6a0c72a742 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:10.281713 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d617cb4-6f51-42e9-bd09-4f6a0c72a742 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:10.294054 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 639/1259] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:42:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:10.300912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933933a5-4285-4791-885f-30f55ba39927 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:10.330776 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:10.421015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933933a5-4285-4791-885f-30f55ba39927 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:10.459408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34869b5-4eb2-4267-9f37-5bf48d9f72f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:10.460198 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34869b5-4eb2-4267-9f37-5bf48d9f72f6 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:10.460476 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34869b5-4eb2-4267-9f37-5bf48d9f72f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:10.460810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34869b5-4eb2-4267-9f37-5bf48d9f72f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:10.461721 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34869b5-4eb2-4267-9f37-5bf48d9f72f6 None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=f4786051321f4a6f80a6cd23eacb09bc) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:10.462041 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34869b5-4eb2-4267-9f37-5bf48d9f72f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:10.463455 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34869b5-4eb2-4267-9f37-5bf48d9f72f6 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:10.469766 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 621/1260] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:42:10 2022] GET /identity/v3/role_assignments?user.id=f4786051321f4a6f80a6cd23eacb09bc&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:12.551080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6bfa0-28e2-4ba4-a0fe-befb9ab34bb6 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:12.551681 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6bfa0-28e2-4ba4-a0fe-befb9ab34bb6 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:12.551681 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6bfa0-28e2-4ba4-a0fe-befb9ab34bb6 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:12.553044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 640/1261] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:42:12.558165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652570f-6647-4d83-be37-77ed6cef1ee2 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:12.558551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652570f-6647-4d83-be37-77ed6cef1ee2 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:12.558710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652570f-6647-4d83-be37-77ed6cef1ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:12.575654 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c652570f-6647-4d83-be37-77ed6cef1ee2 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:12.576861 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c652570f-6647-4d83-be37-77ed6cef1ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:12.642675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 622/1262] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:42:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 20:42:12.665645 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6bfa0-28e2-4ba4-a0fe-befb9ab34bb6 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:12.676413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:12.770475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6bfa0-28e2-4ba4-a0fe-befb9ab34bb6 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:12.803631 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b26be-1ab2-44d7-9f52-32458ccd0d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:12.804439 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b26be-1ab2-44d7-9f52-32458ccd0d84 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:12.804723 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b26be-1ab2-44d7-9f52-32458ccd0d84 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:12.805000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b26be-1ab2-44d7-9f52-32458ccd0d84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:12.815461 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b26be-1ab2-44d7-9f52-32458ccd0d84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-601b26be-1ab2-44d7-9f52-32458ccd0d84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:42:12.823009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:12.824169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:12.825378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:12.825378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:12.825378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:12.825378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:12.825378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:42:12.825378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:42:12.825378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:42:12.825378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:42:12.825378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:42:12.825378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:42:12.825378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:42:12.825378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:42:12.825378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 641/1263] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:42:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:12.830924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652570f-6647-4d83-be37-77ed6cef1ee2 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:12.849163 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:12.935059 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652570f-6647-4d83-be37-77ed6cef1ee2 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:12.968660 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4750eb1-f155-4975-bdf1-0cd16255ee64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:12.969326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4750eb1-f155-4975-bdf1-0cd16255ee64 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:12.969611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4750eb1-f155-4975-bdf1-0cd16255ee64 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:12.969891 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4750eb1-f155-4975-bdf1-0cd16255ee64 None admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:12.971534 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4750eb1-f155-4975-bdf1-0cd16255ee64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:12.971877 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4750eb1-f155-4975-bdf1-0cd16255ee64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:12.973244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4750eb1-f155-4975-bdf1-0cd16255ee64 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:12.981480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 623/1264] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:42:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:12.988075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b26be-1ab2-44d7-9f52-32458ccd0d84 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:13.053432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:13.088943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b26be-1ab2-44d7-9f52-32458ccd0d84 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:13.130750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db796f25-ef22-4bea-b208-1dd6b2097be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:13.131548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db796f25-ef22-4bea-b208-1dd6b2097be4 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:13.131875 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db796f25-ef22-4bea-b208-1dd6b2097be4 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:13.132150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db796f25-ef22-4bea-b208-1dd6b2097be4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:13.143112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db796f25-ef22-4bea-b208-1dd6b2097be4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:13.144596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db796f25-ef22-4bea-b208-1dd6b2097be4 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:13.166876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-db796f25-ef22-4bea-b208-1dd6b2097be4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:13.166876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:13.166876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:13.166876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:13.166876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:13.166876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:13.166876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:13.166876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:13.166876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:13.166876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:13.166876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:13.166876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:13.166876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:42:13.166876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:42:13.166876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:13.168250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:13.169471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:42:13.169471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:42:13.169471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:13.169471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:42:13.169471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 642/1265] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:13.176177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4750eb1-f155-4975-bdf1-0cd16255ee64 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:13.256459 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:13.276189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4750eb1-f155-4975-bdf1-0cd16255ee64 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:13.309388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa12e2-0beb-4a2b-b474-a4fd14055753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:13.310037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa12e2-0beb-4a2b-b474-a4fd14055753 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:13.310320 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa12e2-0beb-4a2b-b474-a4fd14055753 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:13.310594 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa12e2-0beb-4a2b-b474-a4fd14055753 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:13.311304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa12e2-0beb-4a2b-b474-a4fd14055753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:13.311605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa12e2-0beb-4a2b-b474-a4fd14055753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:13.312991 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa12e2-0beb-4a2b-b474-a4fd14055753 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:13.324852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 624/1266] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:13.331223 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db796f25-ef22-4bea-b208-1dd6b2097be4 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:13.429350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db796f25-ef22-4bea-b208-1dd6b2097be4 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:13.448834 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:13.462606 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab059f-a59d-41b4-96a3-843d84ca5713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:13.463388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab059f-a59d-41b4-96a3-843d84ca5713 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:13.463664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab059f-a59d-41b4-96a3-843d84ca5713 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:13.463963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab059f-a59d-41b4-96a3-843d84ca5713 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:13.471245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab059f-a59d-41b4-96a3-843d84ca5713 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:13.472695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab059f-a59d-41b4-96a3-843d84ca5713 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-91ab059f-a59d-41b4-96a3-843d84ca5713 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:42:13.487117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:13.488233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:13.489404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:13.489404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:13.489404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:42:13.489404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:42:13.489404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:42:13.489404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:42:13.489404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:42:13.489404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:42:13.489404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:42:13.489404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:42:13.489404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 643/1267] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:42:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 20:42:13.494284 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa12e2-0beb-4a2b-b474-a4fd14055753 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:13.591865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa12e2-0beb-4a2b-b474-a4fd14055753 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:13.625628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc32134-6b8c-44d0-93eb-4b451ede66a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:13.626314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc32134-6b8c-44d0-93eb-4b451ede66a7 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:13.626667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc32134-6b8c-44d0-93eb-4b451ede66a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:13.627140 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc32134-6b8c-44d0-93eb-4b451ede66a7 None admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:13.628035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc32134-6b8c-44d0-93eb-4b451ede66a7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:13.628340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc32134-6b8c-44d0-93eb-4b451ede66a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:13.629750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc32134-6b8c-44d0-93eb-4b451ede66a7 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:13.638233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:13.647337 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 625/1268] 10.222.0.60 () {56 vars in 1143 bytes} [Thu Jul 21 20:42:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:13.653106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab059f-a59d-41b4-96a3-843d84ca5713 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:13.751900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab059f-a59d-41b4-96a3-843d84ca5713 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:13.786351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c1f72-6aec-4cf2-bcd1-59930bffbaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:13.787126 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c1f72-6aec-4cf2-bcd1-59930bffbaa4 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:13.787409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c1f72-6aec-4cf2-bcd1-59930bffbaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:13.787683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c1f72-6aec-4cf2-bcd1-59930bffbaa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:13.798032 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c1f72-6aec-4cf2-bcd1-59930bffbaa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:13.800194 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c1f72-6aec-4cf2-bcd1-59930bffbaa4 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:13.816778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-c51c1f72-6aec-4cf2-bcd1-59930bffbaa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:13.816778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:13.816778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:13.816778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:13.816778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:13.816778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:13.816778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:13.816778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:13.816778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:13.816778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:13.816778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:13.816778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:13.816778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:42:13.816778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:42:13.816778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:13.817806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:13.819075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:42:13.819075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:42:13.819075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:13.819075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:42:13.819075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 644/1269] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:13.823643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc32134-6b8c-44d0-93eb-4b451ede66a7 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:13.921173 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc32134-6b8c-44d0-93eb-4b451ede66a7 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:13.955288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e848f8-0122-4a61-be03-090262eeeb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:13.955967 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e848f8-0122-4a61-be03-090262eeeb84 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:13.956255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e848f8-0122-4a61-be03-090262eeeb84 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:13.956532 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e848f8-0122-4a61-be03-090262eeeb84 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:13.957253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e848f8-0122-4a61-be03-090262eeeb84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:13.957558 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e848f8-0122-4a61-be03-090262eeeb84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:13.958927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e848f8-0122-4a61-be03-090262eeeb84 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:13.970142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 626/1270] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:13.975948 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c1f72-6aec-4cf2-bcd1-59930bffbaa4 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:14.008790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:14.075122 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c1f72-6aec-4cf2-bcd1-59930bffbaa4 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:14.114635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43921e9-d1d3-4019-9b1b-bcbaa7be9af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:14.115443 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43921e9-d1d3-4019-9b1b-bcbaa7be9af0 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:14.115859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43921e9-d1d3-4019-9b1b-bcbaa7be9af0 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:14.116025 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43921e9-d1d3-4019-9b1b-bcbaa7be9af0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:14.124668 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43921e9-d1d3-4019-9b1b-bcbaa7be9af0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:14.126115 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43921e9-d1d3-4019-9b1b-bcbaa7be9af0 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:14.142434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-b43921e9-d1d3-4019-9b1b-bcbaa7be9af0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:42:14.142434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:14.142434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:14.142434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:14.142434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:14.142434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:14.142434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:14.142434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:14.142434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:14.142434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:14.142434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:14.142434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:14.142434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:42:14.142434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:42:14.142434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:14.143522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:14.144587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:42:14.144587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:42:14.144587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:42:14.144587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:42:14.144587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:42:14.144587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:42:14.144587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:42:14.144587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:42:14.144587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 645/1271] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:42:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:14.150257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e848f8-0122-4a61-be03-090262eeeb84 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:14.210708 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:14.252047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e848f8-0122-4a61-be03-090262eeeb84 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:14.284794 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc34776-b14a-4c83-af73-37e91d2cbb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:14.285445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc34776-b14a-4c83-af73-37e91d2cbb27 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:14.285731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc34776-b14a-4c83-af73-37e91d2cbb27 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:14.286010 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc34776-b14a-4c83-af73-37e91d2cbb27 None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:14.286795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc34776-b14a-4c83-af73-37e91d2cbb27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:14.287106 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc34776-b14a-4c83-af73-37e91d2cbb27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:14.288558 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc34776-b14a-4c83-af73-37e91d2cbb27 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:14.300821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 627/1272] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:42:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:14.306801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43921e9-d1d3-4019-9b1b-bcbaa7be9af0 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:14.398721 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:14.406019 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43921e9-d1d3-4019-9b1b-bcbaa7be9af0 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:14.441887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b871e1c-c51f-447c-920d-b17a56fc6680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:14.442763 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b871e1c-c51f-447c-920d-b17a56fc6680 None admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:14.443045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b871e1c-c51f-447c-920d-b17a56fc6680 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:14.443321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b871e1c-c51f-447c-920d-b17a56fc6680 None admin] PATH_INFO: `/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/f4786051321f4a6f80a6cd23eacb09bc/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:14.445297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b871e1c-c51f-447c-920d-b17a56fc6680 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc61fe15894891a9054911321f4ac3, user_id=f4786051321f4a6f80a6cd23eacb09bc, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:14.446708 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b871e1c-c51f-447c-920d-b17a56fc6680 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:14.459565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 646/1273] 10.222.0.60 () {58 vars in 1439 bytes} [Thu Jul 21 20:42:14 2022] PUT /identity/v3/projects/d6bc61fe15894891a9054911321f4ac3/users/f4786051321f4a6f80a6cd23eacb09bc/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:42:16.497611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43536906-82ee-42e3-a5e6-5555a85d5fa5 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:16.498128 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43536906-82ee-42e3-a5e6-5555a85d5fa5 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:16.498246 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43536906-82ee-42e3-a5e6-5555a85d5fa5 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:16.499532 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 628/1274] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:42:16.506361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d0056-cef9-4918-8759-7a226c6d842a None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:16.506794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d0056-cef9-4918-8759-7a226c6d842a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:16.506959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d0056-cef9-4918-8759-7a226c6d842a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:16.525552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-ec9d0056-cef9-4918-8759-7a226c6d842a None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:16.527122 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ec9d0056-cef9-4918-8759-7a226c6d842a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:16.597074 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 647/1275] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:42:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:42:16.621331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43536906-82ee-42e3-a5e6-5555a85d5fa5 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:16.721607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43536906-82ee-42e3-a5e6-5555a85d5fa5 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:16.738973 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:16.755457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70a910-e848-4a15-bfd7-d4a024eca0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:16.756264 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70a910-e848-4a15-bfd7-d4a024eca0ed None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:16.756547 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70a910-e848-4a15-bfd7-d4a024eca0ed None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:16.756821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70a910-e848-4a15-bfd7-d4a024eca0ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:16.766612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70a910-e848-4a15-bfd7-d4a024eca0ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-ab70a910-e848-4a15-bfd7-d4a024eca0ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 20:42:16.774148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:16.775223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:16.776124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:16.776124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:16.776124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:16.776124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:16.776124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 20:42:16.776124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 20:42:16.776124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 20:42:16.776124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 20:42:16.776124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 20:42:16.776124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 20:42:16.776124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 20:42:16.776124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:16.776124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 629/1276] 10.222.0.60 () {56 vars in 1096 bytes} [Thu Jul 21 20:42:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:16.781357 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d0056-cef9-4918-8759-7a226c6d842a None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:16.863239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:16.881658 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d0056-cef9-4918-8759-7a226c6d842a None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:16.915010 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4a285-aa46-4818-a18a-bcaab83a208c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:16.915665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4a285-aa46-4818-a18a-bcaab83a208c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:16.915969 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4a285-aa46-4818-a18a-bcaab83a208c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:16.916239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4a285-aa46-4818-a18a-bcaab83a208c None admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:16.916973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4a285-aa46-4818-a18a-bcaab83a208c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:16.917276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4a285-aa46-4818-a18a-bcaab83a208c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:16.918622 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e4a285-aa46-4818-a18a-bcaab83a208c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:16.927127 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 648/1277] 10.222.0.60 () {56 vars in 1099 bytes} [Thu Jul 21 20:42:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:16.932854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70a910-e848-4a15-bfd7-d4a024eca0ed None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:17.034208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70a910-e848-4a15-bfd7-d4a024eca0ed None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:17.067295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d68f3-fbab-4590-9c7c-9e31e0571e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:17.068105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d68f3-fbab-4590-9c7c-9e31e0571e53 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:17.068385 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d68f3-fbab-4590-9c7c-9e31e0571e53 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:17.068660 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d68f3-fbab-4590-9c7c-9e31e0571e53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:17.078959 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d68f3-fbab-4590-9c7c-9e31e0571e53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:17.080994 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d68f3-fbab-4590-9c7c-9e31e0571e53 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:17.100270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-b06d68f3-fbab-4590-9c7c-9e31e0571e53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:17.100270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:17.100270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:17.100270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:17.100270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:17.100270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:17.100270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:17.100270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:17.100270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:17.100270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:17.100270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:17.100270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:17.100270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:42:17.100270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:42:17.100270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:17.101359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:17.102266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:42:17.102266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:42:17.102266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:17.102266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:17.102266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 630/1278] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:17.108342 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4a285-aa46-4818-a18a-bcaab83a208c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:17.210485 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e4a285-aa46-4818-a18a-bcaab83a208c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:17.244086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc2221-79a0-4abc-b251-6d661f2f184b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:17.244736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc2221-79a0-4abc-b251-6d661f2f184b None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:17.245018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc2221-79a0-4abc-b251-6d661f2f184b None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:17.245292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc2221-79a0-4abc-b251-6d661f2f184b None admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:17.246000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc2221-79a0-4abc-b251-6d661f2f184b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:17.246304 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc2221-79a0-4abc-b251-6d661f2f184b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:17.247656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc2221-79a0-4abc-b251-6d661f2f184b None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:17.259409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 649/1279] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:17.265115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d68f3-fbab-4590-9c7c-9e31e0571e53 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:17.296900 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:17.366576 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d68f3-fbab-4590-9c7c-9e31e0571e53 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:17.401938 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500d260-cea1-4997-a155-eca0425203bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:17.402285 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500d260-cea1-4997-a155-eca0425203bc None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:17.402569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500d260-cea1-4997-a155-eca0425203bc None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:17.402861 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500d260-cea1-4997-a155-eca0425203bc None admin] PATH_INFO: `/v3/users/placement` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:17.410651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500d260-cea1-4997-a155-eca0425203bc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:17.412116 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500d260-cea1-4997-a155-eca0425203bc None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-5500d260-cea1-4997-a155-eca0425203bc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 20:42:17.426206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:17.427384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:17.428839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:17.428839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:17.428839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 20:42:17.428839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 20:42:17.428839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 20:42:17.428839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 20:42:17.428839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 20:42:17.428839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 20:42:17.428839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 20:42:17.428839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:17.428839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 631/1280] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:42:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:17.433221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc2221-79a0-4abc-b251-6d661f2f184b None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:17.437004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:17.530704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc2221-79a0-4abc-b251-6d661f2f184b None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:17.562803 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebbd0c-e514-4f9a-adb9-d71110fdc78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:17.563396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebbd0c-e514-4f9a-adb9-d71110fdc78c None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:17.564217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebbd0c-e514-4f9a-adb9-d71110fdc78c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:17.564516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebbd0c-e514-4f9a-adb9-d71110fdc78c None admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:17.565335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebbd0c-e514-4f9a-adb9-d71110fdc78c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:17.565638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebbd0c-e514-4f9a-adb9-d71110fdc78c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:17.567037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebbd0c-e514-4f9a-adb9-d71110fdc78c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:17.584036 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 650/1281] 10.222.0.60 () {56 vars in 1143 bytes} [Thu Jul 21 20:42:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:17.589914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500d260-cea1-4997-a155-eca0425203bc None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:17.678645 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:17.688498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500d260-cea1-4997-a155-eca0425203bc None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:17.721390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a2f60-8aed-49ed-9464-53d080428f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:17.722024 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a2f60-8aed-49ed-9464-53d080428f09 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:17.722311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a2f60-8aed-49ed-9464-53d080428f09 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:17.722589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a2f60-8aed-49ed-9464-53d080428f09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:17.788017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a2f60-8aed-49ed-9464-53d080428f09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:17.789474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a2f60-8aed-49ed-9464-53d080428f09 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:17.877609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:17.888175 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 632/1282] 10.222.0.60 () {58 vars in 1282 bytes} [Thu Jul 21 20:42:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 20:42:17.894724 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebbd0c-e514-4f9a-adb9-d71110fdc78c None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:17.958559 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:17.997319 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebbd0c-e514-4f9a-adb9-d71110fdc78c None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:18.030881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577debbb-bf1c-488f-82c1-d1f7bc0b8503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:18.031662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577debbb-bf1c-488f-82c1-d1f7bc0b8503 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:18.031982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577debbb-bf1c-488f-82c1-d1f7bc0b8503 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:18.032259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577debbb-bf1c-488f-82c1-d1f7bc0b8503 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:18.042731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577debbb-bf1c-488f-82c1-d1f7bc0b8503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:18.044198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577debbb-bf1c-488f-82c1-d1f7bc0b8503 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:18.061327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-577debbb-bf1c-488f-82c1-d1f7bc0b8503 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:18.061327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:18.061327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:18.061327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:18.061327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:18.061327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:18.061327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:18.061327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:18.061327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:18.061327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:18.061327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:18.061327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:18.061327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 20:42:18.061327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 20:42:18.061327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:18.062456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:18.063564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 20:42:18.063564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 20:42:18.063564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 20:42:18.063564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:42:18.063564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 651/1283] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:18.068897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a2f60-8aed-49ed-9464-53d080428f09 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:18.176771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a2f60-8aed-49ed-9464-53d080428f09 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:18.211419 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c92ad-daf8-4cfe-8d1b-02eeebbb1673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:18.212114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c92ad-daf8-4cfe-8d1b-02eeebbb1673 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:18.212401 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c92ad-daf8-4cfe-8d1b-02eeebbb1673 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:18.212677 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c92ad-daf8-4cfe-8d1b-02eeebbb1673 None admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:18.213468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c92ad-daf8-4cfe-8d1b-02eeebbb1673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:18.214024 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c92ad-daf8-4cfe-8d1b-02eeebbb1673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:18.216643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c92ad-daf8-4cfe-8d1b-02eeebbb1673 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:18.228635 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 633/1284] 10.222.0.60 () {56 vars in 1109 bytes} [Thu Jul 21 20:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:18.234571 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577debbb-bf1c-488f-82c1-d1f7bc0b8503 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:18.333043 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577debbb-bf1c-488f-82c1-d1f7bc0b8503 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:18.347076 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:18.368161 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6e1a3-515b-47db-b501-faf3a132638d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:18.368936 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6e1a3-515b-47db-b501-faf3a132638d None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:18.369217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6e1a3-515b-47db-b501-faf3a132638d None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:18.369492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6e1a3-515b-47db-b501-faf3a132638d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:18.376015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6e1a3-515b-47db-b501-faf3a132638d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:18.377478 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6e1a3-515b-47db-b501-faf3a132638d None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:18.390867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application [None req-84f6e1a3-515b-47db-b501-faf3a132638d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:42:18.390867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:18.390867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:18.390867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:18.390867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:18.390867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:18.390867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:18.390867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:18.390867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:18.390867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:18.390867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:18.390867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:18.390867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 20:42:18.390867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 20:42:18.390867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:18.391943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:18.393261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 20:42:18.393261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 20:42:18.393261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 20:42:18.393261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 20:42:18.393261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 20:42:18.393261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 20:42:18.393261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 20:42:18.393261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: ERROR keystone.server.flask.application Jul 21 20:42:18.393261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 652/1285] 10.222.0.60 () {56 vars in 1111 bytes} [Thu Jul 21 20:42:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:18.398113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c92ad-daf8-4cfe-8d1b-02eeebbb1673 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:18.447637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:18.497223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c92ad-daf8-4cfe-8d1b-02eeebbb1673 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:18.529670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75aab7-d744-4858-8810-568adb80ffa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:18.530317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75aab7-d744-4858-8810-568adb80ffa7 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:18.530595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75aab7-d744-4858-8810-568adb80ffa7 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:18.530865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75aab7-d744-4858-8810-568adb80ffa7 None admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:18.531642 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75aab7-d744-4858-8810-568adb80ffa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:18.531982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75aab7-d744-4858-8810-568adb80ffa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:18.533978 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75aab7-d744-4858-8810-568adb80ffa7 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:18.545446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 634/1286] 10.222.0.60 () {56 vars in 1148 bytes} [Thu Jul 21 20:42:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:18.551790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6e1a3-515b-47db-b501-faf3a132638d None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:18.655424 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6e1a3-515b-47db-b501-faf3a132638d None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:18.688276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67513e67-ea20-47ca-b44f-d7b3b7fdb304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:18.688955 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67513e67-ea20-47ca-b44f-d7b3b7fdb304 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:18.689238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67513e67-ea20-47ca-b44f-d7b3b7fdb304 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:18.689511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67513e67-ea20-47ca-b44f-d7b3b7fdb304 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:18.690340 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67513e67-ea20-47ca-b44f-d7b3b7fdb304 None admin] RBAC: Adding query filter params (role.id=5e477269dd954c1d9bd08af262e355a1, scope.project.id=d6bc61fe15894891a9054911321f4ac3, user.id=f4786051321f4a6f80a6cd23eacb09bc) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:18.690654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67513e67-ea20-47ca-b44f-d7b3b7fdb304 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:18.692102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67513e67-ea20-47ca-b44f-d7b3b7fdb304 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:18.697895 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 653/1287] 10.222.0.60 () {56 vars in 1374 bytes} [Thu Jul 21 20:42:18 2022] GET /identity/v3/role_assignments?user.id=f4786051321f4a6f80a6cd23eacb09bc&scope.project.id=d6bc61fe15894891a9054911321f4ac3&role.id=5e477269dd954c1d9bd08af262e355a1 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:20.764060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514d497-048b-4bd1-abcc-b83b48cf0208 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:20.764582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514d497-048b-4bd1-abcc-b83b48cf0208 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:20.764582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514d497-048b-4bd1-abcc-b83b48cf0208 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:20.765910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 635/1288] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:42:20.772407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c475d-0775-44b1-b977-53e941e323b4 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:20.772973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c475d-0775-44b1-b977-53e941e323b4 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:20.772973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c475d-0775-44b1-b977-53e941e323b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:20.789699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-765c475d-0775-44b1-b977-53e941e323b4 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:20.790888 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-765c475d-0775-44b1-b977-53e941e323b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:20.820914 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:20.858112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 654/1289] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:42:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:42:20.881451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514d497-048b-4bd1-abcc-b83b48cf0208 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:20.982780 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514d497-048b-4bd1-abcc-b83b48cf0208 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:20.993420 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:21.017274 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e0fb7-b9e0-4ccb-88eb-90d526d8c93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:21.018199 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e0fb7-b9e0-4ccb-88eb-90d526d8c93f None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:21.018483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e0fb7-b9e0-4ccb-88eb-90d526d8c93f None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:21.018758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e0fb7-b9e0-4ccb-88eb-90d526d8c93f None admin] PATH_INFO: `/v3/services/placement` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:21.019404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e0fb7-b9e0-4ccb-88eb-90d526d8c93f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:21.020815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e0fb7-b9e0-4ccb-88eb-90d526d8c93f None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-334e0fb7-b9e0-4ccb-88eb-90d526d8c93f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:21.032366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:21.034166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:21.035309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:21.035309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:21.035309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:21.035309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:21.035309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:21.035309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:21.035309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:21.035309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:21.035309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:21.035309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:21.035309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:42:21.035309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:42:21.035309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:42:21.035309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:21.035309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 636/1290] 10.222.0.60 () {56 vars in 1117 bytes} [Thu Jul 21 20:42:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:21.040486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c475d-0775-44b1-b977-53e941e323b4 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:21.148006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c475d-0775-44b1-b977-53e941e323b4 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:21.185152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295d000-0ff4-49f0-a99b-f860663d7141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:21.185811 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295d000-0ff4-49f0-a99b-f860663d7141 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:21.186093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295d000-0ff4-49f0-a99b-f860663d7141 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:21.186371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295d000-0ff4-49f0-a99b-f860663d7141 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:21.187078 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295d000-0ff4-49f0-a99b-f860663d7141 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:21.187386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295d000-0ff4-49f0-a99b-f860663d7141 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:21.189395 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295d000-0ff4-49f0-a99b-f860663d7141 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:21.194491 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 655/1291] 10.222.0.60 () {56 vars in 1116 bytes} [Thu Jul 21 20:42:21 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:42:21.200305 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e0fb7-b9e0-4ccb-88eb-90d526d8c93f None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:21.299785 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e0fb7-b9e0-4ccb-88eb-90d526d8c93f None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:21.333912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9565749-9c47-4a9f-b271-e036f1076be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:21.334555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9565749-9c47-4a9f-b271-e036f1076be8 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:21.334840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9565749-9c47-4a9f-b271-e036f1076be8 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:21.335115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9565749-9c47-4a9f-b271-e036f1076be8 None admin] PATH_INFO: `/v3/services` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:21.335848 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9565749-9c47-4a9f-b271-e036f1076be8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:21.336155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9565749-9c47-4a9f-b271-e036f1076be8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:21.337513 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9565749-9c47-4a9f-b271-e036f1076be8 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:21.341886 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 637/1292] 10.222.0.60 () {56 vars in 1116 bytes} [Thu Jul 21 20:42:21 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:23.366278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327246f5-8166-4a94-927c-6e421bbd40a0 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:23.368081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327246f5-8166-4a94-927c-6e421bbd40a0 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:23.368081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327246f5-8166-4a94-927c-6e421bbd40a0 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:23.368725 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 656/1293] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:42:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:42:23.373790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da1818-082c-45b8-896c-907eb8d5ffe4 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:23.374216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da1818-082c-45b8-896c-907eb8d5ffe4 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:23.374327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da1818-082c-45b8-896c-907eb8d5ffe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:23.391200 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-24da1818-082c-45b8-896c-907eb8d5ffe4 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:23.392987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-24da1818-082c-45b8-896c-907eb8d5ffe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:23.421599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:23.458660 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 638/1294] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:42:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:42:23.482255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327246f5-8166-4a94-927c-6e421bbd40a0 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:23.513175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:23.585428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327246f5-8166-4a94-927c-6e421bbd40a0 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:23.618581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd72c3-b4d1-4205-ab9a-503969b8fe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:23.619182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd72c3-b4d1-4205-ab9a-503969b8fe3c None admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:23.619453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd72c3-b4d1-4205-ab9a-503969b8fe3c None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:23.619752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd72c3-b4d1-4205-ab9a-503969b8fe3c None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:23.620483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd72c3-b4d1-4205-ab9a-503969b8fe3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:23.621842 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd72c3-b4d1-4205-ab9a-503969b8fe3c None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:23.635381 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 657/1295] 10.222.0.60 () {60 vars in 1141 bytes} [Thu Jul 21 20:42:23 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:42:25.697255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbee82b-a107-4ca0-ae5f-3d7133cfcd52 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:25.697804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbee82b-a107-4ca0-ae5f-3d7133cfcd52 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:25.697804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbee82b-a107-4ca0-ae5f-3d7133cfcd52 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:25.699167 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 639/1296] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:42:25.705064 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6352f12-7d22-4f1b-b415-8908479437c4 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:25.705462 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6352f12-7d22-4f1b-b415-8908479437c4 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:25.705620 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6352f12-7d22-4f1b-b415-8908479437c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:25.722816 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-b6352f12-7d22-4f1b-b415-8908479437c4 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:25.724036 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b6352f12-7d22-4f1b-b415-8908479437c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:25.799422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 658/1297] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:42:25 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:42:25.822284 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbee82b-a107-4ca0-ae5f-3d7133cfcd52 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:25.922865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbee82b-a107-4ca0-ae5f-3d7133cfcd52 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:25.956870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f240e61-4d81-4313-a2ed-41d16013b6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:25.957589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f240e61-4d81-4313-a2ed-41d16013b6cb None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:25.957870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f240e61-4d81-4313-a2ed-41d16013b6cb None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:25.958144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f240e61-4d81-4313-a2ed-41d16013b6cb None admin] PATH_INFO: `/v3/services/placement` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:25.958791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f240e61-4d81-4313-a2ed-41d16013b6cb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:25.960203 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f240e61-4d81-4313-a2ed-41d16013b6cb None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-4f240e61-4d81-4313-a2ed-41d16013b6cb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:25.971673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:25.972966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:25.974039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:25.974039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:25.974039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:25.974039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:25.974039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:25.974039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:25.974039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:25.974039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:25.974039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:25.974039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:25.974039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:42:25.974039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:42:25.974039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:42:25.974039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:25.974039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 640/1298] 10.222.0.60 () {56 vars in 1117 bytes} [Thu Jul 21 20:42:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:25.978950 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6352f12-7d22-4f1b-b415-8908479437c4 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:25.997991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:26.081147 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6352f12-7d22-4f1b-b415-8908479437c4 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:26.124988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79410d2a-cdc9-42d4-b674-b4a8c12d2385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:26.125881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79410d2a-cdc9-42d4-b674-b4a8c12d2385 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:26.126794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79410d2a-cdc9-42d4-b674-b4a8c12d2385 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:26.127129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79410d2a-cdc9-42d4-b674-b4a8c12d2385 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:26.128062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79410d2a-cdc9-42d4-b674-b4a8c12d2385 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:26.128383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79410d2a-cdc9-42d4-b674-b4a8c12d2385 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:26.130040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79410d2a-cdc9-42d4-b674-b4a8c12d2385 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:26.137285 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 659/1299] 10.222.0.60 () {56 vars in 1116 bytes} [Thu Jul 21 20:42:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:26.146670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f240e61-4d81-4313-a2ed-41d16013b6cb None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:26.183966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:26.253396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f240e61-4d81-4313-a2ed-41d16013b6cb None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:26.285652 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f5628-9701-41b3-95ef-6da4559b0c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:26.286308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f5628-9701-41b3-95ef-6da4559b0c1a None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:26.286592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f5628-9701-41b3-95ef-6da4559b0c1a None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:26.286866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f5628-9701-41b3-95ef-6da4559b0c1a None admin] PATH_INFO: `/v3/endpoints` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:26.287651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f5628-9701-41b3-95ef-6da4559b0c1a None admin] RBAC: Adding query filter params (interface=public, service_id=f6019aca135449018336832f66f4ef20, region_id=RegionOne) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:26.287990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f5628-9701-41b3-95ef-6da4559b0c1a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:26.289356 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f5628-9701-41b3-95ef-6da4559b0c1a None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:26.294270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 641/1300] 10.222.0.60 () {56 vars in 1251 bytes} [Thu Jul 21 20:42:26 2022] GET /identity/v3/endpoints?service_id=f6019aca135449018336832f66f4ef20&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:26.300461 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79410d2a-cdc9-42d4-b674-b4a8c12d2385 None admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:26.384133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:26.403810 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79410d2a-cdc9-42d4-b674-b4a8c12d2385 None admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:26.437990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af0f7c-90f9-4007-92ab-7f3fbf56ec14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:26.438676 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af0f7c-90f9-4007-92ab-7f3fbf56ec14 None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:26.438965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af0f7c-90f9-4007-92ab-7f3fbf56ec14 None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:26.439241 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af0f7c-90f9-4007-92ab-7f3fbf56ec14 None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:26.439960 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af0f7c-90f9-4007-92ab-7f3fbf56ec14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:26.441326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af0f7c-90f9-4007-92ab-7f3fbf56ec14 None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:26.448053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 660/1301] 10.222.0.60 () {56 vars in 1087 bytes} [Thu Jul 21 20:42:26 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:42:28.456512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db0e16-78cf-47dc-aaaa-de1a490b415c None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:28.457053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db0e16-78cf-47dc-aaaa-de1a490b415c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:28.457053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db0e16-78cf-47dc-aaaa-de1a490b415c None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:28.458446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 642/1302] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:42:28.465389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc3ba3-db1e-421b-9697-be21cd0267de None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:28.465839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc3ba3-db1e-421b-9697-be21cd0267de None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:28.465956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc3ba3-db1e-421b-9697-be21cd0267de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:28.484189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-00cc3ba3-db1e-421b-9697-be21cd0267de None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:42:28.485387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-00cc3ba3-db1e-421b-9697-be21cd0267de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:42:28.551517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 661/1303] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:42:28 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 20:42:28.575465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db0e16-78cf-47dc-aaaa-de1a490b415c None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:28.668858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:28.676066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db0e16-78cf-47dc-aaaa-de1a490b415c None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:28.707908 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ffb01-d244-4c37-a850-0e71af643642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:28.708688 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ffb01-d244-4c37-a850-0e71af643642 None admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:28.708969 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ffb01-d244-4c37-a850-0e71af643642 None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:28.709243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ffb01-d244-4c37-a850-0e71af643642 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:28.709886 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ffb01-d244-4c37-a850-0e71af643642 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:28.711240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ffb01-d244-4c37-a850-0e71af643642 None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application [None req-3a7ffb01-d244-4c37-a850-0e71af643642 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 20:42:28.723159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 20:42:28.724371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 20:42:28.725413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 20:42:28.725413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application with Lock( Jul 21 20:42:28.725413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 20:42:28.725413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self._enter() Jul 21 20:42:28.725413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 20:42:28.725413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 20:42:28.725413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 20:42:28.725413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application return self.creator() Jul 21 20:42:28.725413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 20:42:28.725413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application created_value = creator( Jul 21 20:42:28.725413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 20:42:28.725413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 20:42:28.725413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 20:42:28.725413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: ERROR keystone.server.flask.application Jul 21 20:42:28.725413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 643/1304] 10.222.0.60 () {56 vars in 1117 bytes} [Thu Jul 21 20:42:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 20:42:28.730511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc3ba3-db1e-421b-9697-be21cd0267de None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:28.808119 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:28.830980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc3ba3-db1e-421b-9697-be21cd0267de None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:28.864984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613973c-5877-43c8-a7c8-37143cb322ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:28.865643 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613973c-5877-43c8-a7c8-37143cb322ed None admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:28.865926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613973c-5877-43c8-a7c8-37143cb322ed None admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:28.866203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613973c-5877-43c8-a7c8-37143cb322ed None admin] PATH_INFO: `/v3/services` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:28.866907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613973c-5877-43c8-a7c8-37143cb322ed None admin] RBAC: Adding query filter params (name=placement) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:42:28.867213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613973c-5877-43c8-a7c8-37143cb322ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:28.868613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613973c-5877-43c8-a7c8-37143cb322ed None admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:28.873660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 662/1305] 10.222.0.60 () {56 vars in 1116 bytes} [Thu Jul 21 20:42:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:42:28.879621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ffb01-d244-4c37-a850-0e71af643642 None admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:42:28.977528 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ffb01-d244-4c37-a850-0e71af643642 None admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:42:29.009840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34165207-35cc-4d13-b9bf-0eeb8d9c520d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:42:29.010476 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34165207-35cc-4d13-b9bf-0eeb8d9c520d None admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:42:29.010755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34165207-35cc-4d13-b9bf-0eeb8d9c520d None admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:42:29.011035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34165207-35cc-4d13-b9bf-0eeb8d9c520d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:42:29.011783 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34165207-35cc-4d13-b9bf-0eeb8d9c520d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:42:29.013141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34165207-35cc-4d13-b9bf-0eeb8d9c520d None admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:42:29.031421 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 644/1306] 10.222.0.60 () {60 vars in 1144 bytes} [Thu Jul 21 20:42:28 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:42:38.870812 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=30 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:42:39.027046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:34.865266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4396a-05c3-4218-adff-1e14deaab5b9 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:34.865856 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4396a-05c3-4218-adff-1e14deaab5b9 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:34.865856 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4396a-05c3-4218-adff-1e14deaab5b9 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:34.867233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 663/1307] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:43:34.872774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a07f08-9bf2-44b1-8dda-89f51d4171a8 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:34.873250 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a07f08-9bf2-44b1-8dda-89f51d4171a8 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:34.873338 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a07f08-9bf2-44b1-8dda-89f51d4171a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:34.989562 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d4a07f08-9bf2-44b1-8dda-89f51d4171a8 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:43:34.991019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d4a07f08-9bf2-44b1-8dda-89f51d4171a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:35.220563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 645/1308] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:43:35.278916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e722af-db78-4eb4-9fdb-dce800d67906 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:35.279287 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e722af-db78-4eb4-9fdb-dce800d67906 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:35.279601 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e722af-db78-4eb4-9fdb-dce800d67906 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:35.281337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 664/1309] 10.222.0.60 () {54 vars in 965 bytes} [Thu Jul 21 20:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:43:35.288958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d1d58-d795-4bc1-8c8f-c5e59d01286b None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:35.289338 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d1d58-d795-4bc1-8c8f-c5e59d01286b None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:35.289677 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d1d58-d795-4bc1-8c8f-c5e59d01286b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:35.483001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-f65d1d58-d795-4bc1-8c8f-c5e59d01286b None None] MFA Rules not processed for user `33aff48933844401b86eb808937e35c4`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:43:35.485847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f65d1d58-d795-4bc1-8c8f-c5e59d01286b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:35.556381 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:35.667565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 646/1310] 10.222.0.60 () {58 vars in 1062 bytes} [Thu Jul 21 20:43:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:43:35.669598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e722af-db78-4eb4-9fdb-dce800d67906 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:43:35.671361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-06e722af-db78-4eb4-9fdb-dce800d67906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:35.909578 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e722af-db78-4eb4-9fdb-dce800d67906 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:43:36.149655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172890c-33fd-410f-a6db-88d70e71522f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:43:36.150570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172890c-33fd-410f-a6db-88d70e71522f service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:36.150847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172890c-33fd-410f-a6db-88d70e71522f service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:36.151282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172890c-33fd-410f-a6db-88d70e71522f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:36.153214 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1172890c-33fd-410f-a6db-88d70e71522f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:36.221110 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:36.288811 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172890c-33fd-410f-a6db-88d70e71522f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:43:36.290273 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172890c-33fd-410f-a6db-88d70e71522f service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:43:36.316922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 665/1311] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:43:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:43:39.132796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1590a96-458b-48a5-a858-7288fb48ee5e None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:39.133341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1590a96-458b-48a5-a858-7288fb48ee5e None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:39.133702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1590a96-458b-48a5-a858-7288fb48ee5e None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:39.136153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 647/1312] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:43:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:43:39.141969 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109df9b7-03de-4ceb-8d40-f58807bc3685 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:39.142485 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109df9b7-03de-4ceb-8d40-f58807bc3685 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:39.142580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109df9b7-03de-4ceb-8d40-f58807bc3685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:39.254423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-109df9b7-03de-4ceb-8d40-f58807bc3685 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:43:39.255676 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-109df9b7-03de-4ceb-8d40-f58807bc3685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:39.354365 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 666/1313] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:43:39.395160 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1590a96-458b-48a5-a858-7288fb48ee5e None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:43:39.396759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f1590a96-458b-48a5-a858-7288fb48ee5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:39.524308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1590a96-458b-48a5-a858-7288fb48ee5e None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:43:39.630786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:39.634432 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf320add-e977-4f54-99e9-b61832e3085a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:43:39.635891 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf320add-e977-4f54-99e9-b61832e3085a service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:39.636421 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf320add-e977-4f54-99e9-b61832e3085a service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:39.636983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf320add-e977-4f54-99e9-b61832e3085a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:39.639055 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-bf320add-e977-4f54-99e9-b61832e3085a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:39.751725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf320add-e977-4f54-99e9-b61832e3085a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:43:39.753194 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf320add-e977-4f54-99e9-b61832e3085a service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:43:39.773496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 648/1314] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:43:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:43:42.153755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94362f41-1b31-4340-9edf-380f242ca96e None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:42.154297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94362f41-1b31-4340-9edf-380f242ca96e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:42.154297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94362f41-1b31-4340-9edf-380f242ca96e None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:42.155694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 667/1315] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:43:42.161940 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7deb884-0678-4ed1-8955-af7ad6fed7b5 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:42.162400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7deb884-0678-4ed1-8955-af7ad6fed7b5 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:42.162512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7deb884-0678-4ed1-8955-af7ad6fed7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:42.226583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-e7deb884-0678-4ed1-8955-af7ad6fed7b5 None None] MFA Rules not processed for user `805b77c8bb67450db935957c46dc8b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:43:42.227802 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e7deb884-0678-4ed1-8955-af7ad6fed7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:42.279865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:42.325947 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 649/1316] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:43:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:43:42.380799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945bbad-b42c-4474-a534-2a22447377bc None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:42.381085 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945bbad-b42c-4474-a534-2a22447377bc None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:42.381367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945bbad-b42c-4474-a534-2a22447377bc None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:42.382738 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 668/1317] 10.222.0.60 () {54 vars in 965 bytes} [Thu Jul 21 20:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:43:42.388638 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee954c-fe6f-4bfe-a06a-015caa05d69e None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:42.388929 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee954c-fe6f-4bfe-a06a-015caa05d69e None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:42.389208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee954c-fe6f-4bfe-a06a-015caa05d69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:42.466337 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-dcee954c-fe6f-4bfe-a06a-015caa05d69e None None] MFA Rules not processed for user `33aff48933844401b86eb808937e35c4`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:43:42.467548 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-dcee954c-fe6f-4bfe-a06a-015caa05d69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:42.543369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 650/1318] 10.222.0.60 () {58 vars in 1062 bytes} [Thu Jul 21 20:43:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:43:42.552790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945bbad-b42c-4474-a534-2a22447377bc None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:43:42.554314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1945bbad-b42c-4474-a534-2a22447377bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:42.564192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:42.661351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945bbad-b42c-4474-a534-2a22447377bc None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:43:42.740412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2534e71-21ef-4c26-8035-9dc3cfec22ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:43:42.740972 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2534e71-21ef-4c26-8035-9dc3cfec22ba service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:42.741255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2534e71-21ef-4c26-8035-9dc3cfec22ba service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:42.741527 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2534e71-21ef-4c26-8035-9dc3cfec22ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:42.743171 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c2534e71-21ef-4c26-8035-9dc3cfec22ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:42.807687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:42.856424 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2534e71-21ef-4c26-8035-9dc3cfec22ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:43:42.858126 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2534e71-21ef-4c26-8035-9dc3cfec22ba service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:43:42.876755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 669/1319] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:43:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:43:45.904017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81129ff7-4f70-47ea-8ec3-2b98951a6e0f None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:45.904618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81129ff7-4f70-47ea-8ec3-2b98951a6e0f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:45.904618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81129ff7-4f70-47ea-8ec3-2b98951a6e0f None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:45.905973 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 651/1320] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:43:45.911922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3ea03-1205-4c3b-8061-7afe234435db None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:45.912344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3ea03-1205-4c3b-8061-7afe234435db None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:45.912488 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3ea03-1205-4c3b-8061-7afe234435db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:45.979596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-a1b3ea03-1205-4c3b-8061-7afe234435db None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:43:45.980926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a1b3ea03-1205-4c3b-8061-7afe234435db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:46.077865 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 670/1321] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:43:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:43:46.119206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81129ff7-4f70-47ea-8ec3-2b98951a6e0f None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:43:46.120829 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-81129ff7-4f70-47ea-8ec3-2b98951a6e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:46.172426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:46.230396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81129ff7-4f70-47ea-8ec3-2b98951a6e0f None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:43:46.305219 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ac62e-a995-4c4c-b89e-6056ee483092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:43:46.305868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ac62e-a995-4c4c-b89e-6056ee483092 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:46.306153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ac62e-a995-4c4c-b89e-6056ee483092 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:46.306430 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ac62e-a995-4c4c-b89e-6056ee483092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:46.308140 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7d3ac62e-a995-4c4c-b89e-6056ee483092 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:46.405640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:46.420683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ac62e-a995-4c4c-b89e-6056ee483092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:43:46.422246 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ac62e-a995-4c4c-b89e-6056ee483092 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:43:46.447248 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 652/1322] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:43:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:43:49.277154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f19f1-421d-4f8e-a0a1-4f06fd163e8b None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:49.277726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f19f1-421d-4f8e-a0a1-4f06fd163e8b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:49.277726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f19f1-421d-4f8e-a0a1-4f06fd163e8b None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:49.279132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 671/1323] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:43:49.285921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164ee2f-8bdf-4837-984d-1cf886828170 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:49.286495 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164ee2f-8bdf-4837-984d-1cf886828170 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:49.286626 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164ee2f-8bdf-4837-984d-1cf886828170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:49.365279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-2164ee2f-8bdf-4837-984d-1cf886828170 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:43:49.366686 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-2164ee2f-8bdf-4837-984d-1cf886828170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:49.472271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 653/1324] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:43:49.510225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f19f1-421d-4f8e-a0a1-4f06fd163e8b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:43:49.511910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-2e0f19f1-421d-4f8e-a0a1-4f06fd163e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:49.536299 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:49.632785 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f19f1-421d-4f8e-a0a1-4f06fd163e8b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:43:49.706956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a256a76-f9fd-40d5-94de-de341a15125f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:43:49.707609 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a256a76-f9fd-40d5-94de-de341a15125f service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:49.707920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a256a76-f9fd-40d5-94de-de341a15125f service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:49.708195 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a256a76-f9fd-40d5-94de-de341a15125f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:49.709870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1a256a76-f9fd-40d5-94de-de341a15125f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:49.783053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:49.822339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a256a76-f9fd-40d5-94de-de341a15125f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:43:49.823791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a256a76-f9fd-40d5-94de-de341a15125f service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:43:49.842731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 672/1325] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:43:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:43:51.981109 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa09423-b622-4bb8-8a15-bb530c6c28b2 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:51.981622 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa09423-b622-4bb8-8a15-bb530c6c28b2 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:51.981622 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa09423-b622-4bb8-8a15-bb530c6c28b2 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:51.983299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 654/1326] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:43:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:43:51.989983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef3608-f1ba-4ccc-ab8f-611f4da58f87 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:51.990445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef3608-f1ba-4ccc-ab8f-611f4da58f87 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:51.990539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef3608-f1ba-4ccc-ab8f-611f4da58f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:52.055422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-69ef3608-f1ba-4ccc-ab8f-611f4da58f87 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:43:52.056654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-69ef3608-f1ba-4ccc-ab8f-611f4da58f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:52.161008 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 673/1327] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:43:52.197914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa09423-b622-4bb8-8a15-bb530c6c28b2 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:43:52.199437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-eaa09423-b622-4bb8-8a15-bb530c6c28b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:52.261476 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:52.307017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa09423-b622-4bb8-8a15-bb530c6c28b2 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:43:52.389147 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b9a15-b2c4-44ca-b93d-a4ccedaaf137 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:43:52.389792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b9a15-b2c4-44ca-b93d-a4ccedaaf137 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:52.390071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b9a15-b2c4-44ca-b93d-a4ccedaaf137 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:52.390963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b9a15-b2c4-44ca-b93d-a4ccedaaf137 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:52.392937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d58b9a15-b2c4-44ca-b93d-a4ccedaaf137 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:52.503517 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:52.506236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b9a15-b2c4-44ca-b93d-a4ccedaaf137 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:43:52.508103 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b9a15-b2c4-44ca-b93d-a4ccedaaf137 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:43:52.536345 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 655/1328] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:43:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:43:54.667871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915ba0f-1595-434f-9e2f-cb4faeccd3b1 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:54.668391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915ba0f-1595-434f-9e2f-cb4faeccd3b1 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:54.668391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915ba0f-1595-434f-9e2f-cb4faeccd3b1 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:54.669790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 674/1329] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:43:54.675299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5c05e-de33-4d48-8de4-3415b3325d93 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:54.675822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5c05e-de33-4d48-8de4-3415b3325d93 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:54.675923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5c05e-de33-4d48-8de4-3415b3325d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:54.744855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-a9f5c05e-de33-4d48-8de4-3415b3325d93 None None] MFA Rules not processed for user `805b77c8bb67450db935957c46dc8b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:43:54.746060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a9f5c05e-de33-4d48-8de4-3415b3325d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:54.848043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 656/1330] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:43:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:43:54.886063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915ba0f-1595-434f-9e2f-cb4faeccd3b1 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:43:54.887884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-5915ba0f-1595-434f-9e2f-cb4faeccd3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:54.921538 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:54.996068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915ba0f-1595-434f-9e2f-cb4faeccd3b1 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:43:55.070322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3717e-afd0-462d-9acc-70df2835e26b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:43:55.070971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3717e-afd0-462d-9acc-70df2835e26b service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:55.071250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3717e-afd0-462d-9acc-70df2835e26b service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:55.071522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3717e-afd0-462d-9acc-70df2835e26b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:55.073187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6bb3717e-afd0-462d-9acc-70df2835e26b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:55.155936 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:55.200929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3717e-afd0-462d-9acc-70df2835e26b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:43:55.202407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3717e-afd0-462d-9acc-70df2835e26b service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:43:55.220548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 675/1331] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:43:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:43:57.549680 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb79436-422c-462d-afee-d3e417f625e8 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:57.550180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb79436-422c-462d-afee-d3e417f625e8 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:57.550293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb79436-422c-462d-afee-d3e417f625e8 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:57.551615 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 657/1332] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:43:57.557043 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f280732-64d0-4828-a11c-e2dfa1410fab None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:57.557467 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f280732-64d0-4828-a11c-e2dfa1410fab None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:57.557608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f280732-64d0-4828-a11c-e2dfa1410fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:57.622456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-0f280732-64d0-4828-a11c-e2dfa1410fab None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:43:57.623673 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-0f280732-64d0-4828-a11c-e2dfa1410fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:57.734284 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 676/1333] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:43:57.770216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb79436-422c-462d-afee-d3e417f625e8 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:43:57.771910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-0eb79436-422c-462d-afee-d3e417f625e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:57.842464 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:57.878157 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb79436-422c-462d-afee-d3e417f625e8 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:43:57.958461 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ed472-6342-47bc-8e15-21ab10115ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:43:57.959114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ed472-6342-47bc-8e15-21ab10115ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:43:57.959399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ed472-6342-47bc-8e15-21ab10115ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:43:57.959676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ed472-6342-47bc-8e15-21ab10115ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:43:57.961357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d01ed472-6342-47bc-8e15-21ab10115ae4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:43:58.069943 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ed472-6342-47bc-8e15-21ab10115ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:43:58.071304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ed472-6342-47bc-8e15-21ab10115ae4 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:43:58.081850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:43:58.098867 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 658/1334] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:43:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:00.211367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22166a8f-3919-4844-8d69-fb78acd96f7a None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:00.211959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22166a8f-3919-4844-8d69-fb78acd96f7a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:00.211959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22166a8f-3919-4844-8d69-fb78acd96f7a None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:00.213339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 677/1335] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:00.219301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e70203-0c84-487b-82f1-8f388e0bcb08 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:00.219762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e70203-0c84-487b-82f1-8f388e0bcb08 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:00.220048 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e70203-0c84-487b-82f1-8f388e0bcb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:00.285961 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-57e70203-0c84-487b-82f1-8f388e0bcb08 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:00.287168 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-57e70203-0c84-487b-82f1-8f388e0bcb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:00.387814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 659/1336] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:00.429750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22166a8f-3919-4844-8d69-fb78acd96f7a None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:00.431314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-22166a8f-3919-4844-8d69-fb78acd96f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:00.471838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:00.534748 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22166a8f-3919-4844-8d69-fb78acd96f7a None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:00.606760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfead8a5-9e06-40be-8980-e4870316c1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:00.607400 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfead8a5-9e06-40be-8980-e4870316c1ee service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:00.607684 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfead8a5-9e06-40be-8980-e4870316c1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:00.608242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfead8a5-9e06-40be-8980-e4870316c1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:00.610042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-dfead8a5-9e06-40be-8980-e4870316c1ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:00.692957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:00.718433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfead8a5-9e06-40be-8980-e4870316c1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:00.720176 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfead8a5-9e06-40be-8980-e4870316c1ee service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:00.738665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 678/1337] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:03.290783 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f645d3-480a-455d-9fe1-e1e8b3a199a6 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:03.291366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f645d3-480a-455d-9fe1-e1e8b3a199a6 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:03.291366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f645d3-480a-455d-9fe1-e1e8b3a199a6 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:03.292744 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 660/1338] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:03.298168 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b729f-3a2b-434b-be91-98df8ab8a847 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:03.298642 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b729f-3a2b-434b-be91-98df8ab8a847 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:03.298748 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b729f-3a2b-434b-be91-98df8ab8a847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:03.364418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e48b729f-3a2b-434b-be91-98df8ab8a847 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:03.365662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e48b729f-3a2b-434b-be91-98df8ab8a847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:03.469606 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 679/1339] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:03.506360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f645d3-480a-455d-9fe1-e1e8b3a199a6 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:03.507971 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f9f645d3-480a-455d-9fe1-e1e8b3a199a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:03.592468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:03.621757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f645d3-480a-455d-9fe1-e1e8b3a199a6 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:03.694932 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8ab9b-553e-4cfe-b8ef-ba0f16253846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:03.695586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8ab9b-553e-4cfe-b8ef-ba0f16253846 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:03.695945 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8ab9b-553e-4cfe-b8ef-ba0f16253846 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:03.696226 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8ab9b-553e-4cfe-b8ef-ba0f16253846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:03.697888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-13a8ab9b-553e-4cfe-b8ef-ba0f16253846 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:03.810356 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8ab9b-553e-4cfe-b8ef-ba0f16253846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:03.811824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8ab9b-553e-4cfe-b8ef-ba0f16253846 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:03.830297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 661/1340] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:08.037572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65815a5c-da44-44fc-8f58-a199607fc8a1 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:08.038204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65815a5c-da44-44fc-8f58-a199607fc8a1 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:08.038204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65815a5c-da44-44fc-8f58-a199607fc8a1 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:08.039485 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 680/1341] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:08.044993 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb6774-4916-4d66-b1ee-b21ef1f4cf97 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:08.045482 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb6774-4916-4d66-b1ee-b21ef1f4cf97 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:08.045587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb6774-4916-4d66-b1ee-b21ef1f4cf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:08.050360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:08.116974 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-2efb6774-4916-4d66-b1ee-b21ef1f4cf97 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:08.118175 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-2efb6774-4916-4d66-b1ee-b21ef1f4cf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:08.227814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 662/1342] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:08.269787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65815a5c-da44-44fc-8f58-a199607fc8a1 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:08.271345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-65815a5c-da44-44fc-8f58-a199607fc8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:08.326928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:08.383636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65815a5c-da44-44fc-8f58-a199607fc8a1 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:08.462949 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2732a-1504-4935-aa79-7fc5cf6a4d8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:08.463598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2732a-1504-4935-aa79-7fc5cf6a4d8a service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:08.463924 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2732a-1504-4935-aa79-7fc5cf6a4d8a service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:08.464206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2732a-1504-4935-aa79-7fc5cf6a4d8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:08.465852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-80e2732a-1504-4935-aa79-7fc5cf6a4d8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:08.563759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:08.580506 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2732a-1504-4935-aa79-7fc5cf6a4d8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:08.582416 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2732a-1504-4935-aa79-7fc5cf6a4d8a service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:08.600569 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 681/1343] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:11.346369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ae936-e80b-4895-9a0d-dad210720567 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:11.346999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ae936-e80b-4895-9a0d-dad210720567 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:11.346999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ae936-e80b-4895-9a0d-dad210720567 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:11.348414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 663/1344] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:11.355146 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978409ac-88b6-415e-8ef6-f1fa5aa8e96d None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:11.355702 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978409ac-88b6-415e-8ef6-f1fa5aa8e96d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:11.355941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978409ac-88b6-415e-8ef6-f1fa5aa8e96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:11.429940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-978409ac-88b6-415e-8ef6-f1fa5aa8e96d None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:11.431150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-978409ac-88b6-415e-8ef6-f1fa5aa8e96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:11.546078 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 682/1345] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:44:11.585258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ae936-e80b-4895-9a0d-dad210720567 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:11.586814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-566ae936-e80b-4895-9a0d-dad210720567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:11.685134 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:11.704063 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ae936-e80b-4895-9a0d-dad210720567 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:11.783274 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e9f2e-24ef-46ff-a4f3-19a89f8eeddb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:11.783970 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e9f2e-24ef-46ff-a4f3-19a89f8eeddb service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:11.784252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e9f2e-24ef-46ff-a4f3-19a89f8eeddb service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:11.784526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e9f2e-24ef-46ff-a4f3-19a89f8eeddb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:11.786280 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b38e9f2e-24ef-46ff-a4f3-19a89f8eeddb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:11.909599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e9f2e-24ef-46ff-a4f3-19a89f8eeddb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:11.912197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e9f2e-24ef-46ff-a4f3-19a89f8eeddb service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:11.935276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 664/1346] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:16.146409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83833511-19d1-4065-a52f-0ece5597444c None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:16.146830 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83833511-19d1-4065-a52f-0ece5597444c None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:16.147152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83833511-19d1-4065-a52f-0ece5597444c None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:16.149097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 683/1347] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:16.155004 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68739fa-c5ae-46e4-b618-9471b9e98c30 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:16.155523 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68739fa-c5ae-46e4-b618-9471b9e98c30 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:16.155523 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68739fa-c5ae-46e4-b618-9471b9e98c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:16.171417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:16.227218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d68739fa-c5ae-46e4-b618-9471b9e98c30 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:16.228503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d68739fa-c5ae-46e4-b618-9471b9e98c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:16.330816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 665/1348] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:16.370274 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83833511-19d1-4065-a52f-0ece5597444c None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:16.372125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-83833511-19d1-4065-a52f-0ece5597444c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:16.442480 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:16.500574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83833511-19d1-4065-a52f-0ece5597444c None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:16.583548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478f30a-aacc-46c3-a441-230d7054446e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:16.585273 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478f30a-aacc-46c3-a441-230d7054446e service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:16.585273 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478f30a-aacc-46c3-a441-230d7054446e service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:16.585273 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478f30a-aacc-46c3-a441-230d7054446e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:16.586434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-4478f30a-aacc-46c3-a441-230d7054446e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:16.710248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:16.714286 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478f30a-aacc-46c3-a441-230d7054446e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:16.715298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478f30a-aacc-46c3-a441-230d7054446e service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:16.735603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 684/1349] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:21.728221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f278ab8-4e5a-4081-b6aa-7dbaa7e92108 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:21.728667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f278ab8-4e5a-4081-b6aa-7dbaa7e92108 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:21.728870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f278ab8-4e5a-4081-b6aa-7dbaa7e92108 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:21.730690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 666/1350] 10.222.0.60 () {54 vars in 921 bytes} [Thu Jul 21 20:44:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:21.737433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f302e3-34bd-408b-97b0-e639be4c6e64 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:21.737861 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f302e3-34bd-408b-97b0-e639be4c6e64 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:21.738038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f302e3-34bd-408b-97b0-e639be4c6e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:21.862499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d1f302e3-34bd-408b-97b0-e639be4c6e64 None None] MFA Rules not processed for user `f4786051321f4a6f80a6cd23eacb09bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:21.863738 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d1f302e3-34bd-408b-97b0-e639be4c6e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:22.004754 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 685/1351] 10.222.0.60 () {58 vars in 1018 bytes} [Thu Jul 21 20:44:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:44:26.167468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240a0d5-9183-44c7-bba0-7b5835b65391 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:26.168360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240a0d5-9183-44c7-bba0-7b5835b65391 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:26.168809 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240a0d5-9183-44c7-bba0-7b5835b65391 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:26.170359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 667/1352] 10.222.0.60 () {54 vars in 921 bytes} [Thu Jul 21 20:44:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:26.177309 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cc90b-7205-484f-b35b-0c1583c08919 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:26.178162 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cc90b-7205-484f-b35b-0c1583c08919 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:26.178655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cc90b-7205-484f-b35b-0c1583c08919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:26.244983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:26.274538 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-a20cc90b-7205-484f-b35b-0c1583c08919 None None] MFA Rules not processed for user `f4786051321f4a6f80a6cd23eacb09bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:26.275809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a20cc90b-7205-484f-b35b-0c1583c08919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:26.328834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=59 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:26.393727 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 686/1353] 10.222.0.60 () {58 vars in 1018 bytes} [Thu Jul 21 20:44:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:27.920282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5622b69-17bc-4d55-83a6-a379e70a8230 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:27.920876 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5622b69-17bc-4d55-83a6-a379e70a8230 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:27.920876 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5622b69-17bc-4d55-83a6-a379e70a8230 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:27.922814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 668/1354] 10.222.0.60 () {54 vars in 921 bytes} [Thu Jul 21 20:44:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:27.929033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f750d-85c3-4251-899b-809fdc42aef3 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:27.929629 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f750d-85c3-4251-899b-809fdc42aef3 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:27.930160 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f750d-85c3-4251-899b-809fdc42aef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:28.022510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-421f750d-85c3-4251-899b-809fdc42aef3 None None] MFA Rules not processed for user `f4786051321f4a6f80a6cd23eacb09bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:28.023746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-421f750d-85c3-4251-899b-809fdc42aef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:28.067456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:28.136707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 687/1355] 10.222.0.60 () {58 vars in 1018 bytes} [Thu Jul 21 20:44:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:31.588000 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90ae52-c1f8-41fc-a859-03695fa37bfc None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:31.588857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90ae52-c1f8-41fc-a859-03695fa37bfc None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:31.589337 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90ae52-c1f8-41fc-a859-03695fa37bfc None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:31.591824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 669/1356] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:31.598535 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03358cfb-2bf9-416d-90ab-7c1d319ead31 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:31.599538 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03358cfb-2bf9-416d-90ab-7c1d319ead31 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:31.600296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03358cfb-2bf9-416d-90ab-7c1d319ead31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:31.685776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-03358cfb-2bf9-416d-90ab-7c1d319ead31 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:31.687655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-03358cfb-2bf9-416d-90ab-7c1d319ead31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:31.797736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 688/1357] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:31.854326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1af67-27a9-4a57-8bb4-ab4ca29f3612 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:31.855003 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1af67-27a9-4a57-8bb4-ab4ca29f3612 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:31.855528 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1af67-27a9-4a57-8bb4-ab4ca29f3612 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:31.858079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 670/1358] 10.222.0.60 () {54 vars in 962 bytes} [Thu Jul 21 20:44:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:31.869156 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092f0f4-80d5-47d5-899b-9abc0794111f None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:31.869460 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092f0f4-80d5-47d5-899b-9abc0794111f None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:31.869747 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092f0f4-80d5-47d5-899b-9abc0794111f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:31.888885 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:31.980103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-7092f0f4-80d5-47d5-899b-9abc0794111f None None] MFA Rules not processed for user `77812b356c974d2b80102217956169db`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:31.981631 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-7092f0f4-80d5-47d5-899b-9abc0794111f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:32.116240 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 689/1359] 10.222.0.60 () {58 vars in 1059 bytes} [Thu Jul 21 20:44:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:32.121783 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1af67-27a9-4a57-8bb4-ab4ca29f3612 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:32.123370 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-81a1af67-27a9-4a57-8bb4-ab4ca29f3612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:32.291330 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1af67-27a9-4a57-8bb4-ab4ca29f3612 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:32.380935 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:32.428998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750270e-eb21-404f-baa6-9f8c39276f1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:32.429802 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750270e-eb21-404f-baa6-9f8c39276f1d service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:32.430182 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750270e-eb21-404f-baa6-9f8c39276f1d service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:32.430575 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750270e-eb21-404f-baa6-9f8c39276f1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:32.432444 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e750270e-eb21-404f-baa6-9f8c39276f1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:32.539934 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750270e-eb21-404f-baa6-9f8c39276f1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:32.541640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750270e-eb21-404f-baa6-9f8c39276f1d service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:32.563287 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 671/1360] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:34.857337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ec53d-bc48-40ac-a500-45453f71148d None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:34.858055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ec53d-bc48-40ac-a500-45453f71148d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:34.858055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ec53d-bc48-40ac-a500-45453f71148d None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:34.859750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 690/1361] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:34.864894 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a8643-dc95-4ae2-b1c9-f27a0e6e929e None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:34.865315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a8643-dc95-4ae2-b1c9-f27a0e6e929e None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:34.865458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a8643-dc95-4ae2-b1c9-f27a0e6e929e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:34.939619 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-a08a8643-dc95-4ae2-b1c9-f27a0e6e929e None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:34.940860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a08a8643-dc95-4ae2-b1c9-f27a0e6e929e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:34.985021 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:35.064170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 672/1362] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:35.110779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ec53d-bc48-40ac-a500-45453f71148d None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:35.112450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b55ec53d-bc48-40ac-a500-45453f71148d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:35.190724 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:35.308574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ec53d-bc48-40ac-a500-45453f71148d None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:35.439020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89371f-67da-4db1-b3eb-016084b5e2f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:35.439799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89371f-67da-4db1-b3eb-016084b5e2f9 service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:35.440150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89371f-67da-4db1-b3eb-016084b5e2f9 service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:35.440495 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89371f-67da-4db1-b3eb-016084b5e2f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:35.442411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ea89371f-67da-4db1-b3eb-016084b5e2f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:35.468445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:35.546150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89371f-67da-4db1-b3eb-016084b5e2f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:35.547779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89371f-67da-4db1-b3eb-016084b5e2f9 service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:35.569331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 691/1363] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:37.997020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b439dbb-217c-48e0-b09f-9ac4cbfe8aee None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:37.997884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b439dbb-217c-48e0-b09f-9ac4cbfe8aee None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:37.998344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b439dbb-217c-48e0-b09f-9ac4cbfe8aee None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:38.000033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 673/1364] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:38.005510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1df676-502b-41a1-939d-9dca5c90deff None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:38.006331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1df676-502b-41a1-939d-9dca5c90deff None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:38.006793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1df676-502b-41a1-939d-9dca5c90deff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:38.074976 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-ab1df676-502b-41a1-939d-9dca5c90deff None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:38.076557 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ab1df676-502b-41a1-939d-9dca5c90deff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:38.190210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:38.204014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 692/1365] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:38.240783 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77075d52-2eaf-4c5b-bf06-b4faf22f9d28 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:38.241077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77075d52-2eaf-4c5b-bf06-b4faf22f9d28 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:38.241361 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77075d52-2eaf-4c5b-bf06-b4faf22f9d28 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:38.242739 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 674/1366] 10.222.0.60 () {54 vars in 962 bytes} [Thu Jul 21 20:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:38.251098 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299db64-2313-4054-8c5a-4013fef5cab3 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:38.251384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299db64-2313-4054-8c5a-4013fef5cab3 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:38.251592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299db64-2313-4054-8c5a-4013fef5cab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:38.333460 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-f299db64-2313-4054-8c5a-4013fef5cab3 None None] MFA Rules not processed for user `77812b356c974d2b80102217956169db`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:38.334671 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f299db64-2313-4054-8c5a-4013fef5cab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:38.479341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 693/1367] 10.222.0.60 () {58 vars in 1059 bytes} [Thu Jul 21 20:44:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:38.488421 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77075d52-2eaf-4c5b-bf06-b4faf22f9d28 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:38.490014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-77075d52-2eaf-4c5b-bf06-b4faf22f9d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:38.651987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77075d52-2eaf-4c5b-bf06-b4faf22f9d28 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:38.674584 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:38.776388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dde48f-470b-4d6a-ba2e-a236f590c0bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:38.777047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dde48f-470b-4d6a-ba2e-a236f590c0bd service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:38.777327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dde48f-470b-4d6a-ba2e-a236f590c0bd service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:38.777598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dde48f-470b-4d6a-ba2e-a236f590c0bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:38.779241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c7dde48f-470b-4d6a-ba2e-a236f590c0bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:38.882094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dde48f-470b-4d6a-ba2e-a236f590c0bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:38.883727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dde48f-470b-4d6a-ba2e-a236f590c0bd service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:38.904727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 675/1368] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:39.899284 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62653c-3b3e-461c-910e-df117d2f9f56 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:39.899885 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62653c-3b3e-461c-910e-df117d2f9f56 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:39.900214 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62653c-3b3e-461c-910e-df117d2f9f56 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:39.902471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 694/1369] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:39.908805 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2884ec-5ec0-4768-bf7b-824a5441e9ab None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:39.909333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2884ec-5ec0-4768-bf7b-824a5441e9ab None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:39.909501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2884ec-5ec0-4768-bf7b-824a5441e9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:39.946269 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:39.989467 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-ff2884ec-5ec0-4768-bf7b-824a5441e9ab None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:39.991018 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ff2884ec-5ec0-4768-bf7b-824a5441e9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:40.128148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 676/1370] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:40.521203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7ca6e-ce95-491a-a501-4d52bea1bdca None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:40.521615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7ca6e-ce95-491a-a501-4d52bea1bdca None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:40.522020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7ca6e-ce95-491a-a501-4d52bea1bdca None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:40.523912 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 695/1371] 10.222.0.60 () {54 vars in 964 bytes} [Thu Jul 21 20:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:40.530392 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61e1be-27b5-41db-95a6-c4914eaa9a85 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:40.530749 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61e1be-27b5-41db-95a6-c4914eaa9a85 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:40.530963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61e1be-27b5-41db-95a6-c4914eaa9a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:40.621204 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-ab61e1be-27b5-41db-95a6-c4914eaa9a85 None None] MFA Rules not processed for user `a8eef7746fc4417e8d4de0b052f3eaa0`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:40.622426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ab61e1be-27b5-41db-95a6-c4914eaa9a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:40.678015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:40.709808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 677/1372] 10.222.0.60 () {58 vars in 1061 bytes} [Thu Jul 21 20:44:40 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:40.717257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7ca6e-ce95-491a-a501-4d52bea1bdca None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:40.719356 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fec7ca6e-ce95-491a-a501-4d52bea1bdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:40.781187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:40.846798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7ca6e-ce95-491a-a501-4d52bea1bdca None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:40.928860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79c800-8d19-4d4e-b518-e58382b44f85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:40.929519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79c800-8d19-4d4e-b518-e58382b44f85 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:40.929720 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79c800-8d19-4d4e-b518-e58382b44f85 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:40.930068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79c800-8d19-4d4e-b518-e58382b44f85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:40.931647 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-dc79c800-8d19-4d4e-b518-e58382b44f85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:41.043546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:41.060482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79c800-8d19-4d4e-b518-e58382b44f85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:41.062641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79c800-8d19-4d4e-b518-e58382b44f85 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:41.084612 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 696/1373] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:41.175266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818eab3-6257-4115-a174-5e41e011e611 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:41.175756 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818eab3-6257-4115-a174-5e41e011e611 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:41.175845 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818eab3-6257-4115-a174-5e41e011e611 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:41.177239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 678/1374] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:41.182580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6d1ec-10af-4d9f-af8f-8a0bbad51357 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:41.182871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6d1ec-10af-4d9f-af8f-8a0bbad51357 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:41.183149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6d1ec-10af-4d9f-af8f-8a0bbad51357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:41.266556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-a8f6d1ec-10af-4d9f-af8f-8a0bbad51357 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:41.268016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a8f6d1ec-10af-4d9f-af8f-8a0bbad51357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:41.380265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 697/1375] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:41.428009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818eab3-6257-4115-a174-5e41e011e611 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:41.429573 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8818eab3-6257-4115-a174-5e41e011e611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:41.593061 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818eab3-6257-4115-a174-5e41e011e611 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:41.645666 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:41.710764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa90f8-baa3-47f0-b097-8674fb2095f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:41.711335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa90f8-baa3-47f0-b097-8674fb2095f3 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:41.711616 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa90f8-baa3-47f0-b097-8674fb2095f3 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:41.711921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa90f8-baa3-47f0-b097-8674fb2095f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:41.713569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6ffa90f8-baa3-47f0-b097-8674fb2095f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:41.811007 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa90f8-baa3-47f0-b097-8674fb2095f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:41.813290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa90f8-baa3-47f0-b097-8674fb2095f3 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:41.832045 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 679/1376] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:43.916994 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b8955-6a21-488f-a135-72e81d74122a None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:43.917499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b8955-6a21-488f-a135-72e81d74122a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:43.917593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b8955-6a21-488f-a135-72e81d74122a None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:43.918912 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 698/1377] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:43.924187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df36761-c859-4dbd-b7ad-a50d1e8b07bc None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:43.924598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df36761-c859-4dbd-b7ad-a50d1e8b07bc None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:43.924746 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df36761-c859-4dbd-b7ad-a50d1e8b07bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:43.991303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-7df36761-c859-4dbd-b7ad-a50d1e8b07bc None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:43.992572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7df36761-c859-4dbd-b7ad-a50d1e8b07bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:43.999940 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:44.104089 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 680/1378] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:44.158764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b8955-6a21-488f-a135-72e81d74122a None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:44.160310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-cd8b8955-6a21-488f-a135-72e81d74122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:44.224004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:44.312747 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b8955-6a21-488f-a135-72e81d74122a None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:44.429561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80524b07-6500-4949-839e-fb40c67f08bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:44.430214 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80524b07-6500-4949-839e-fb40c67f08bd service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:44.430502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80524b07-6500-4949-839e-fb40c67f08bd service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:44.430783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80524b07-6500-4949-839e-fb40c67f08bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:44.432516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-80524b07-6500-4949-839e-fb40c67f08bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:44.453044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:44.540431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80524b07-6500-4949-839e-fb40c67f08bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:44.542066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80524b07-6500-4949-839e-fb40c67f08bd service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:44.568138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 699/1379] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:46.695050 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0825d0d-39aa-441b-bf8b-2b008d81ce9f None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:46.695654 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0825d0d-39aa-441b-bf8b-2b008d81ce9f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:46.695654 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0825d0d-39aa-441b-bf8b-2b008d81ce9f None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:46.697340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 681/1380] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:46.704067 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093d7fd-75b1-4cb4-812f-87ad7a315a97 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:46.704648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093d7fd-75b1-4cb4-812f-87ad7a315a97 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:46.704648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093d7fd-75b1-4cb4-812f-87ad7a315a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:46.779976 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-4093d7fd-75b1-4cb4-812f-87ad7a315a97 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:46.781589 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-4093d7fd-75b1-4cb4-812f-87ad7a315a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:46.887884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:46.895765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 700/1381] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:46.939645 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0825d0d-39aa-441b-bf8b-2b008d81ce9f None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:46.941328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b0825d0d-39aa-441b-bf8b-2b008d81ce9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:47.119859 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:47.132669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0825d0d-39aa-441b-bf8b-2b008d81ce9f None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:47.258896 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294f868-16d8-40a0-a0f3-50ebbf4da906 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:47.259540 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294f868-16d8-40a0-a0f3-50ebbf4da906 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:47.259977 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294f868-16d8-40a0-a0f3-50ebbf4da906 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:47.260258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294f868-16d8-40a0-a0f3-50ebbf4da906 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:47.261924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e294f868-16d8-40a0-a0f3-50ebbf4da906 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:47.361783 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294f868-16d8-40a0-a0f3-50ebbf4da906 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:47.363400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294f868-16d8-40a0-a0f3-50ebbf4da906 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:47.387770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:47.398832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 682/1382] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:49.570211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d54dbe-dc14-4d16-9af6-5c1ba9197b54 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:49.570887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d54dbe-dc14-4d16-9af6-5c1ba9197b54 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:49.570887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d54dbe-dc14-4d16-9af6-5c1ba9197b54 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:49.572400 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 701/1383] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:49.579590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f334fc6-19e8-4aff-9326-24d7fbc0f1db None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:49.580116 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f334fc6-19e8-4aff-9326-24d7fbc0f1db None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:49.580232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f334fc6-19e8-4aff-9326-24d7fbc0f1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:49.651495 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-9f334fc6-19e8-4aff-9326-24d7fbc0f1db None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:49.652906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-9f334fc6-19e8-4aff-9326-24d7fbc0f1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:49.777000 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 683/1384] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:44:49.830006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d54dbe-dc14-4d16-9af6-5c1ba9197b54 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:49.831670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-06d54dbe-dc14-4d16-9af6-5c1ba9197b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:50.031530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d54dbe-dc14-4d16-9af6-5c1ba9197b54 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:50.124797 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:50.183893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93224f-7a68-42c7-b229-cf7925508c70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:50.184750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93224f-7a68-42c7-b229-cf7925508c70 service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:50.185095 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93224f-7a68-42c7-b229-cf7925508c70 service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:50.185446 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93224f-7a68-42c7-b229-cf7925508c70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:50.188916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ae93224f-7a68-42c7-b229-cf7925508c70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:50.285511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93224f-7a68-42c7-b229-cf7925508c70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:50.287046 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93224f-7a68-42c7-b229-cf7925508c70 service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:50.306561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 702/1385] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:52.496585 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fd7ae-453f-4036-b1ce-cb903e87db79 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:52.497227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fd7ae-453f-4036-b1ce-cb903e87db79 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:52.497227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fd7ae-453f-4036-b1ce-cb903e87db79 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:52.498770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 684/1386] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:52.504536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36adc4-8d9e-43f7-b68c-01c2c9d8a821 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:52.505014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36adc4-8d9e-43f7-b68c-01c2c9d8a821 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:52.505106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36adc4-8d9e-43f7-b68c-01c2c9d8a821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:52.578674 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-1a36adc4-8d9e-43f7-b68c-01c2c9d8a821 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:52.580225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1a36adc4-8d9e-43f7-b68c-01c2c9d8a821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:52.618554 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:52.688242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 703/1387] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:52.732754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fd7ae-453f-4036-b1ce-cb903e87db79 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:52.735112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-399fd7ae-453f-4036-b1ce-cb903e87db79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:52.823042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:52.900017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fd7ae-453f-4036-b1ce-cb903e87db79 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:53.019336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98e6b4-dcee-497f-9b81-2eb6a44a461a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:53.019917 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98e6b4-dcee-497f-9b81-2eb6a44a461a service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:53.020197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98e6b4-dcee-497f-9b81-2eb6a44a461a service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:53.020468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98e6b4-dcee-497f-9b81-2eb6a44a461a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:53.022131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-dd98e6b4-dcee-497f-9b81-2eb6a44a461a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:53.065433 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:53.126446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98e6b4-dcee-497f-9b81-2eb6a44a461a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:53.128234 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98e6b4-dcee-497f-9b81-2eb6a44a461a service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:53.148105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 685/1388] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:55.391695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906011e-42cf-4bc5-b138-3bdcf6835c14 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:55.392282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906011e-42cf-4bc5-b138-3bdcf6835c14 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:55.392282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906011e-42cf-4bc5-b138-3bdcf6835c14 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:55.393701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 704/1389] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:55.399542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da239ac2-e50b-4cca-973b-5b740fd15a19 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:55.400261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da239ac2-e50b-4cca-973b-5b740fd15a19 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:55.400261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da239ac2-e50b-4cca-973b-5b740fd15a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:55.474920 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-da239ac2-e50b-4cca-973b-5b740fd15a19 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:55.476164 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-da239ac2-e50b-4cca-973b-5b740fd15a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:55.582624 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 686/1390] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:55.631906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906011e-42cf-4bc5-b138-3bdcf6835c14 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:55.633466 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-5906011e-42cf-4bc5-b138-3bdcf6835c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:55.786558 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906011e-42cf-4bc5-b138-3bdcf6835c14 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:55.802971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:55.904327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccce0d8-418c-4a73-8318-7b0c4ba6ada0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:55.905062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccce0d8-418c-4a73-8318-7b0c4ba6ada0 service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:55.905343 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccce0d8-418c-4a73-8318-7b0c4ba6ada0 service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:55.905619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccce0d8-418c-4a73-8318-7b0c4ba6ada0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:55.907263 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-cccce0d8-418c-4a73-8318-7b0c4ba6ada0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:55.998444 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccce0d8-418c-4a73-8318-7b0c4ba6ada0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:56.000766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccce0d8-418c-4a73-8318-7b0c4ba6ada0 service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:56.018962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 705/1391] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:56.594738 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20279f55-429e-4ba1-b2a5-135e807a0fd5 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:56.595303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20279f55-429e-4ba1-b2a5-135e807a0fd5 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:56.595303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20279f55-429e-4ba1-b2a5-135e807a0fd5 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:56.596719 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 687/1392] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:56.602111 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c33177-2a78-4dd3-ad3a-e56bf42fe010 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:56.602499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c33177-2a78-4dd3-ad3a-e56bf42fe010 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:56.602773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c33177-2a78-4dd3-ad3a-e56bf42fe010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:56.628865 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:56.671160 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-89c33177-2a78-4dd3-ad3a-e56bf42fe010 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:56.672510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-89c33177-2a78-4dd3-ad3a-e56bf42fe010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:56.775701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 706/1393] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:56.832629 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2684a-6bff-4ab4-8707-8b055f32c580 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:56.833072 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2684a-6bff-4ab4-8707-8b055f32c580 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:56.833359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2684a-6bff-4ab4-8707-8b055f32c580 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:56.834743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 688/1394] 10.222.0.60 () {54 vars in 966 bytes} [Thu Jul 21 20:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:56.839847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5685e-e3c1-443a-9a61-3b6caa2e1696 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:56.840127 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5685e-e3c1-443a-9a61-3b6caa2e1696 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:56.840409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5685e-e3c1-443a-9a61-3b6caa2e1696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:56.920121 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-77a5685e-e3c1-443a-9a61-3b6caa2e1696 None None] MFA Rules not processed for user `1d596de177e24ca5a254bda6644a07da`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:56.922184 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-77a5685e-e3c1-443a-9a61-3b6caa2e1696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:56.964373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:57.004898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 707/1395] 10.222.0.60 () {58 vars in 1063 bytes} [Thu Jul 21 20:44:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:57.012655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2684a-6bff-4ab4-8707-8b055f32c580 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:57.014234 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7ec2684a-6bff-4ab4-8707-8b055f32c580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:57.028370 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:57.134495 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2684a-6bff-4ab4-8707-8b055f32c580 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:57.215505 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5e0f8-ce7f-462b-b421-4d6fc2d4c2fb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d596de177e24ca5a254bda6644a07da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:57.216108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5e0f8-ce7f-462b-b421-4d6fc2d4c2fb service glance] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:57.216398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5e0f8-ce7f-462b-b421-4d6fc2d4c2fb service glance] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:57.216676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5e0f8-ce7f-462b-b421-4d6fc2d4c2fb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:57.219059 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-aae5e0f8-ce7f-462b-b421-4d6fc2d4c2fb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:57.288104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:57.331460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5e0f8-ce7f-462b-b421-4d6fc2d4c2fb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:57.332950 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5e0f8-ce7f-462b-b421-4d6fc2d4c2fb service glance] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:57.352327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 689/1396] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:44:58.112137 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bdfb1a-0f26-4f30-95f2-0098cbab6a7b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:58.112672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bdfb1a-0f26-4f30-95f2-0098cbab6a7b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:58.112822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bdfb1a-0f26-4f30-95f2-0098cbab6a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:58.156082 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef4ad6-4627-4e0e-9ef4-e06b9a567a01 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:58.156676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef4ad6-4627-4e0e-9ef4-e06b9a567a01 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:58.156676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef4ad6-4627-4e0e-9ef4-e06b9a567a01 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:58.157951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 690/1397] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:44:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:44:58.164570 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a09d8-f149-48a4-ab23-11d8cc792402 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:58.164974 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a09d8-f149-48a4-ab23-11d8cc792402 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:58.165333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a09d8-f149-48a4-ab23-11d8cc792402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:58.192515 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-16bdfb1a-0f26-4f30-95f2-0098cbab6a7b None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:58.194006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-16bdfb1a-0f26-4f30-95f2-0098cbab6a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:58.234832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-a71a09d8-f149-48a4-ab23-11d8cc792402 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:58.236119 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a71a09d8-f149-48a4-ab23-11d8cc792402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:58.286505 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 708/1398] 10.222.0.60 () {58 vars in 1009 bytes} [Thu Jul 21 20:44:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:44:58.330591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c28385-2152-4d31-97dd-e33735dc0e83 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:58.330881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c28385-2152-4d31-97dd-e33735dc0e83 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:58.331161 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c28385-2152-4d31-97dd-e33735dc0e83 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:58.332588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 709/1399] 10.222.0.60 () {54 vars in 943 bytes} [Thu Jul 21 20:44:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:44:58.338125 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 691/1400] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:44:58.339327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cc3d4-627b-4cd7-a0eb-561ec8ae6ed9 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:58.339327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cc3d4-627b-4cd7-a0eb-561ec8ae6ed9 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:58.339524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cc3d4-627b-4cd7-a0eb-561ec8ae6ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:58.383348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a09d8-f149-48a4-ab23-11d8cc792402 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:58.384599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a71a09d8-f149-48a4-ab23-11d8cc792402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:58.410524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:58.417144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:58.419077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-ca4cc3d4-627b-4cd7-a0eb-561ec8ae6ed9 None None] MFA Rules not processed for user `833a474f828f4349bea0d1649461c77e`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:44:58.420352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ca4cc3d4-627b-4cd7-a0eb-561ec8ae6ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:58.508329 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a09d8-f149-48a4-ab23-11d8cc792402 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:58.529650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 710/1401] 10.222.0.60 () {58 vars in 1040 bytes} [Thu Jul 21 20:44:58 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:44:58.542851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cc3d4-627b-4cd7-a0eb-561ec8ae6ed9 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:44:58.553951 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cc3d4-627b-4cd7-a0eb-561ec8ae6ed9 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:44:58.630093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc389b-f8ff-48bd-b3b0-842e90397d50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:58.631111 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc389b-f8ff-48bd-b3b0-842e90397d50 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:58.631484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc389b-f8ff-48bd-b3b0-842e90397d50 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:58.631898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc389b-f8ff-48bd-b3b0-842e90397d50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:58.633954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-00cc389b-f8ff-48bd-b3b0-842e90397d50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:58.653401 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:58.663428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc78f5-fa82-4cff-9ce5-fdb7a6cfacc9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:44:58.664106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc78f5-fa82-4cff-9ce5-fdb7a6cfacc9 service swift] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:44:58.664385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc78f5-fa82-4cff-9ce5-fdb7a6cfacc9 service swift] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:44:58.664664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc78f5-fa82-4cff-9ce5-fdb7a6cfacc9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:44:58.666334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e4cc78f5-fa82-4cff-9ce5-fdb7a6cfacc9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:44:58.668859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:44:58.728412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc389b-f8ff-48bd-b3b0-842e90397d50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:58.730035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc389b-f8ff-48bd-b3b0-842e90397d50 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:58.749346 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 692/1402] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:44:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:44:58.753212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc78f5-fa82-4cff-9ce5-fdb7a6cfacc9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:44:58.755045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc78f5-fa82-4cff-9ce5-fdb7a6cfacc9 service swift] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:44:58.763296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 711/1403] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 20:44:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 20:45:00.890750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd132d-695c-40ce-ab12-0f36eda7627b None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:00.891330 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd132d-695c-40ce-ab12-0f36eda7627b None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:00.891668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd132d-695c-40ce-ab12-0f36eda7627b None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:00.893563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 693/1404] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:00.898662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770e9b9-4f7f-4a17-83d3-246d97a97d97 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:00.899560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770e9b9-4f7f-4a17-83d3-246d97a97d97 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:00.899953 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770e9b9-4f7f-4a17-83d3-246d97a97d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:00.993272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-9770e9b9-4f7f-4a17-83d3-246d97a97d97 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:00.994951 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9770e9b9-4f7f-4a17-83d3-246d97a97d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:01.194945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:01.224257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 712/1405] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:45:01.269433 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd132d-695c-40ce-ab12-0f36eda7627b None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:01.270964 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-0bdd132d-695c-40ce-ab12-0f36eda7627b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:01.433164 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:01.435484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd132d-695c-40ce-ab12-0f36eda7627b None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:01.579091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbce4c-bf0c-4a58-a083-b3c91b964cce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:01.579784 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbce4c-bf0c-4a58-a083-b3c91b964cce service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:01.580077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbce4c-bf0c-4a58-a083-b3c91b964cce service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:01.580358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbce4c-bf0c-4a58-a083-b3c91b964cce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:01.582064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b9bbce4c-bf0c-4a58-a083-b3c91b964cce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:01.683554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbce4c-bf0c-4a58-a083-b3c91b964cce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:01.685180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbce4c-bf0c-4a58-a083-b3c91b964cce service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:01.710099 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 694/1406] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:03.978357 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f22231-7090-48f5-8cbe-6a53ef852d28 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:03.978932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f22231-7090-48f5-8cbe-6a53ef852d28 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:03.978932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f22231-7090-48f5-8cbe-6a53ef852d28 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:03.980497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 713/1407] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:03.987053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5dcd8-7262-43f5-9331-936729b2860b None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:03.988166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5dcd8-7262-43f5-9331-936729b2860b None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:03.988461 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5dcd8-7262-43f5-9331-936729b2860b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:03.991939 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:04.074816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-74e5dcd8-7262-43f5-9331-936729b2860b None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:04.077261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-74e5dcd8-7262-43f5-9331-936729b2860b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:04.201747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 695/1408] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:04.254817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f22231-7090-48f5-8cbe-6a53ef852d28 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:04.256388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f9f22231-7090-48f5-8cbe-6a53ef852d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:04.438037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f22231-7090-48f5-8cbe-6a53ef852d28 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:04.524674 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:04.589899 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7919d41-5dbb-49d7-bd6b-a217789eab6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:04.590983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7919d41-5dbb-49d7-bd6b-a217789eab6a service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:04.591266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7919d41-5dbb-49d7-bd6b-a217789eab6a service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:04.591542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7919d41-5dbb-49d7-bd6b-a217789eab6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:04.593865 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b7919d41-5dbb-49d7-bd6b-a217789eab6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:04.703121 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7919d41-5dbb-49d7-bd6b-a217789eab6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:04.704776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7919d41-5dbb-49d7-bd6b-a217789eab6a service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:04.725157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 714/1409] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:14.199922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=64 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:14.722175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=64 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:21.809475 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0bf5c-9f43-436b-8997-816a9ca5d695 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:21.809998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0bf5c-9f43-436b-8997-816a9ca5d695 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:21.809998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0bf5c-9f43-436b-8997-816a9ca5d695 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:21.811928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 696/1410] 10.222.0.60 () {54 vars in 918 bytes} [Thu Jul 21 20:45:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:21.817152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db20192-7abd-46e3-b972-8695186b5447 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:21.817548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db20192-7abd-46e3-b972-8695186b5447 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:21.817732 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db20192-7abd-46e3-b972-8695186b5447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:21.895429 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-4db20192-7abd-46e3-b972-8695186b5447 None None] MFA Rules not processed for user `f4786051321f4a6f80a6cd23eacb09bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:21.896688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-4db20192-7abd-46e3-b972-8695186b5447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:22.003723 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 715/1411] 10.222.0.60 () {58 vars in 1015 bytes} [Thu Jul 21 20:45:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:22.018582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127de6e-f920-4f70-8c25-e4bfb03040b0 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:22.018874 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127de6e-f920-4f70-8c25-e4bfb03040b0 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:22.019151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127de6e-f920-4f70-8c25-e4bfb03040b0 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:22.020595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 697/1412] 10.222.0.60 () {54 vars in 918 bytes} [Thu Jul 21 20:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:22.025213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099f1d3-8167-4dda-8623-3dd14adc1796 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:22.025504 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099f1d3-8167-4dda-8623-3dd14adc1796 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:22.025782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099f1d3-8167-4dda-8623-3dd14adc1796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:22.097692 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:22.100506 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-4099f1d3-8167-4dda-8623-3dd14adc1796 None None] MFA Rules not processed for user `77812b356c974d2b80102217956169db`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:22.102464 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-4099f1d3-8167-4dda-8623-3dd14adc1796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:22.219451 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 716/1413] 10.222.0.60 () {58 vars in 1015 bytes} [Thu Jul 21 20:45:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:24.769024 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c363a54-77d4-4595-a5a8-4a998c94d4c5 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:24.769535 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c363a54-77d4-4595-a5a8-4a998c94d4c5 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:24.769535 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c363a54-77d4-4595-a5a8-4a998c94d4c5 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:24.770908 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:24.776498 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285e57e-a1bf-4188-8515-4af496c609c1 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:24.776930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285e57e-a1bf-4188-8515-4af496c609c1 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:24.777062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285e57e-a1bf-4188-8515-4af496c609c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:24.844370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-6285e57e-a1bf-4188-8515-4af496c609c1 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:24.845595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6285e57e-a1bf-4188-8515-4af496c609c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:24.901316 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:24.945231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:24.989256 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c363a54-77d4-4595-a5a8-4a998c94d4c5 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:24.990754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-0c363a54-77d4-4595-a5a8-4a998c94d4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:25.140552 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c363a54-77d4-4595-a5a8-4a998c94d4c5 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:25.213576 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:25.256697 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983cb4f-c956-430f-9576-c335333e5df5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:25.257347 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983cb4f-c956-430f-9576-c335333e5df5 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:25.257624 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983cb4f-c956-430f-9576-c335333e5df5 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:25.257895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983cb4f-c956-430f-9576-c335333e5df5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:25.259520 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c983cb4f-c956-430f-9576-c335333e5df5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:25.350526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983cb4f-c956-430f-9576-c335333e5df5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:25.352174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983cb4f-c956-430f-9576-c335333e5df5 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:25.370722 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 699/1416] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:26.277499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7e9b6-8438-421b-ae86-1522189136c2 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:26.278114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7e9b6-8438-421b-ae86-1522189136c2 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:26.278114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7e9b6-8438-421b-ae86-1522189136c2 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:26.280190 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:26.285234 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6abbe-517e-43bf-b007-2136f38ff0cf None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:26.285624 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6abbe-517e-43bf-b007-2136f38ff0cf None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:26.285788 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6abbe-517e-43bf-b007-2136f38ff0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:26.360489 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-02b6abbe-517e-43bf-b007-2136f38ff0cf None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:26.361694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-02b6abbe-517e-43bf-b007-2136f38ff0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:26.388406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:26.464395 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:26.619789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7e9b6-8438-421b-ae86-1522189136c2 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:26.621317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-76c7e9b6-8438-421b-ae86-1522189136c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:26.768572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7e9b6-8438-421b-ae86-1522189136c2 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:26.810102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:26.890206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7919bfd-96df-429e-8da8-c517daa3b25d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:26.890761 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7919bfd-96df-429e-8da8-c517daa3b25d service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:26.891042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7919bfd-96df-429e-8da8-c517daa3b25d service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:26.891314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7919bfd-96df-429e-8da8-c517daa3b25d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:26.892988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e7919bfd-96df-429e-8da8-c517daa3b25d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:26.982555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7919bfd-96df-429e-8da8-c517daa3b25d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:26.984830 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7919bfd-96df-429e-8da8-c517daa3b25d service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:27.003294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 719/1419] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:27.886747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4ca41-02e1-4974-a454-85f24a73c049 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:27.887313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4ca41-02e1-4974-a454-85f24a73c049 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:27.887313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4ca41-02e1-4974-a454-85f24a73c049 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:27.888835 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:27.893850 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31284be1-23e1-40ba-98ff-30e4878617b4 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:27.894273 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31284be1-23e1-40ba-98ff-30e4878617b4 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:27.894409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31284be1-23e1-40ba-98ff-30e4878617b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:27.947095 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:27.969081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-31284be1-23e1-40ba-98ff-30e4878617b4 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:27.970285 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-31284be1-23e1-40ba-98ff-30e4878617b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:28.069415 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:45:28.224358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4ca41-02e1-4974-a454-85f24a73c049 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:28.226008 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-bea4ca41-02e1-4974-a454-85f24a73c049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:28.372197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4ca41-02e1-4974-a454-85f24a73c049 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:28.384308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:28.486258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be889a-539b-4683-87eb-a66ce6a95c04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:28.486910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be889a-539b-4683-87eb-a66ce6a95c04 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:28.487195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be889a-539b-4683-87eb-a66ce6a95c04 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:28.487472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be889a-539b-4683-87eb-a66ce6a95c04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:28.489168 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d3be889a-539b-4683-87eb-a66ce6a95c04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:28.588939 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be889a-539b-4683-87eb-a66ce6a95c04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:28.590551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be889a-539b-4683-87eb-a66ce6a95c04 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:28.609184 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 702/1422] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:29.472365 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a5349-c5a0-467c-973b-95dabb50ff38 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:29.472921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a5349-c5a0-467c-973b-95dabb50ff38 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:29.472921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a5349-c5a0-467c-973b-95dabb50ff38 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:29.474315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:29.479218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a009a8-abbf-4e57-b633-8b89a85e716b None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:29.479639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a009a8-abbf-4e57-b633-8b89a85e716b None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:29.479797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a009a8-abbf-4e57-b633-8b89a85e716b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:29.499579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:29.555947 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d9a009a8-abbf-4e57-b633-8b89a85e716b None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:29.557152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d9a009a8-abbf-4e57-b633-8b89a85e716b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:29.665156 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:29.815289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a5349-c5a0-467c-973b-95dabb50ff38 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:29.816920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-007a5349-c5a0-467c-973b-95dabb50ff38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:29.927672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:29.964440 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a5349-c5a0-467c-973b-95dabb50ff38 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:30.080021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df77e59-6995-44a7-ac8b-51c53859fee8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:30.080670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df77e59-6995-44a7-ac8b-51c53859fee8 service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:30.080956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df77e59-6995-44a7-ac8b-51c53859fee8 service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:30.081233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df77e59-6995-44a7-ac8b-51c53859fee8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:30.082878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8df77e59-6995-44a7-ac8b-51c53859fee8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:30.160789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:30.176553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df77e59-6995-44a7-ac8b-51c53859fee8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:30.178218 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df77e59-6995-44a7-ac8b-51c53859fee8 service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:30.196247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 722/1425] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:31.059334 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1006650-3b8e-41ce-99a8-c0ceb63466e8 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:31.059936 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1006650-3b8e-41ce-99a8-c0ceb63466e8 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:31.059936 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1006650-3b8e-41ce-99a8-c0ceb63466e8 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:31.061446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:31.066533 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79b79f-2d82-49cc-a4c9-33bc4f78023e None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:31.066934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79b79f-2d82-49cc-a4c9-33bc4f78023e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:31.067099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79b79f-2d82-49cc-a4c9-33bc4f78023e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:31.148623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-fc79b79f-2d82-49cc-a4c9-33bc4f78023e None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:31.149839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fc79b79f-2d82-49cc-a4c9-33bc4f78023e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:31.251495 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:31.405940 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1006650-3b8e-41ce-99a8-c0ceb63466e8 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:31.407475 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a1006650-3b8e-41ce-99a8-c0ceb63466e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:31.491396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:31.553517 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1006650-3b8e-41ce-99a8-c0ceb63466e8 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:31.669655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f9b7f-17bc-4f28-9191-ec51adc79d3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:31.670557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f9b7f-17bc-4f28-9191-ec51adc79d3a service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:31.670885 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f9b7f-17bc-4f28-9191-ec51adc79d3a service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:31.671153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f9b7f-17bc-4f28-9191-ec51adc79d3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:31.672873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-589f9b7f-17bc-4f28-9191-ec51adc79d3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:31.721372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:31.762153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f9b7f-17bc-4f28-9191-ec51adc79d3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:31.764452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f9b7f-17bc-4f28-9191-ec51adc79d3a service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:31.782182 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 705/1428] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:32.604166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad41e9f-2109-4f45-b795-dc0b490f7a1c None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:32.604803 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad41e9f-2109-4f45-b795-dc0b490f7a1c None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:32.604803 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad41e9f-2109-4f45-b795-dc0b490f7a1c None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:32.606118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:32.612421 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce1d72-feac-4058-9a00-15e4c1eb6383 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:32.612831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce1d72-feac-4058-9a00-15e4c1eb6383 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:32.612982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce1d72-feac-4058-9a00-15e4c1eb6383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:32.690838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-7dce1d72-feac-4058-9a00-15e4c1eb6383 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:32.692025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7dce1d72-feac-4058-9a00-15e4c1eb6383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:32.795690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:32.948639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad41e9f-2109-4f45-b795-dc0b490f7a1c None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:32.950117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8ad41e9f-2109-4f45-b795-dc0b490f7a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:33.000506 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:33.099286 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad41e9f-2109-4f45-b795-dc0b490f7a1c None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:33.214470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a782b-0caf-4fb9-ac1a-2506ce40f804 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:33.215028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a782b-0caf-4fb9-ac1a-2506ce40f804 service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:33.215309 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a782b-0caf-4fb9-ac1a-2506ce40f804 service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:33.215583 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a782b-0caf-4fb9-ac1a-2506ce40f804 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:33.217444 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-954a782b-0caf-4fb9-ac1a-2506ce40f804 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:33.224289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:33.308169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a782b-0caf-4fb9-ac1a-2506ce40f804 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:33.309780 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a782b-0caf-4fb9-ac1a-2506ce40f804 service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:33.329543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 725/1431] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:34.145052 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcd2d8-57ed-4591-b5d0-3b1f433879f4 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:34.145627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcd2d8-57ed-4591-b5d0-3b1f433879f4 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:34.145627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcd2d8-57ed-4591-b5d0-3b1f433879f4 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:34.147015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:34.152079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e73739-bcee-492c-988a-d5fa81fec0af None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:34.152540 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e73739-bcee-492c-988a-d5fa81fec0af None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:34.152661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e73739-bcee-492c-988a-d5fa81fec0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:34.228422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e2e73739-bcee-492c-988a-d5fa81fec0af None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:34.229647 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e2e73739-bcee-492c-988a-d5fa81fec0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:34.297764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:34.328500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:34.485818 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efcd2d8-57ed-4591-b5d0-3b1f433879f4 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:34.487349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5efcd2d8-57ed-4591-b5d0-3b1f433879f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:34.505011 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:34.633982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efcd2d8-57ed-4591-b5d0-3b1f433879f4 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:34.721199 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:34.747269 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101a528-f5ba-40a0-87d9-30717037d033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:34.748522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101a528-f5ba-40a0-87d9-30717037d033 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:34.748869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101a528-f5ba-40a0-87d9-30717037d033 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:34.749214 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101a528-f5ba-40a0-87d9-30717037d033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:34.750892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1101a528-f5ba-40a0-87d9-30717037d033 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:34.841122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1101a528-f5ba-40a0-87d9-30717037d033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:34.842842 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1101a528-f5ba-40a0-87d9-30717037d033 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:34.861004 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 708/1434] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:35.691781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3670df-ee59-4529-a523-d91b9700877a None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:35.692295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3670df-ee59-4529-a523-d91b9700877a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:35.692295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3670df-ee59-4529-a523-d91b9700877a None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:35.693652 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:35.698509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddbc8e-5e1a-40ae-9c7f-12cb048adecc None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:35.698944 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddbc8e-5e1a-40ae-9c7f-12cb048adecc None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:35.699081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddbc8e-5e1a-40ae-9c7f-12cb048adecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:35.776997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-53ddbc8e-5e1a-40ae-9c7f-12cb048adecc None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:35.778219 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-53ddbc8e-5e1a-40ae-9c7f-12cb048adecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:35.821648 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:35.877465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:36.029283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3670df-ee59-4529-a523-d91b9700877a None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:36.030749 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-2c3670df-ee59-4529-a523-d91b9700877a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:36.180974 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3670df-ee59-4529-a523-d91b9700877a None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:36.236217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:36.294268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51b423-f0c6-4161-bcfe-d23153a8f213 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:36.295397 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51b423-f0c6-4161-bcfe-d23153a8f213 service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:36.295679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51b423-f0c6-4161-bcfe-d23153a8f213 service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:36.295989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51b423-f0c6-4161-bcfe-d23153a8f213 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:36.297625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6b51b423-f0c6-4161-bcfe-d23153a8f213 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:36.391466 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51b423-f0c6-4161-bcfe-d23153a8f213 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:36.393138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51b423-f0c6-4161-bcfe-d23153a8f213 service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:36.413700 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 728/1437] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:37.225683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd7db2-2497-40c8-be48-ccdd2b9ade1c None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:37.226265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd7db2-2497-40c8-be48-ccdd2b9ade1c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:37.226265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd7db2-2497-40c8-be48-ccdd2b9ade1c None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:37.227639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:37.232561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd617a6-2323-4f4d-8b61-df018b572c03 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:37.233084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd617a6-2323-4f4d-8b61-df018b572c03 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:37.233195 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd617a6-2323-4f4d-8b61-df018b572c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:37.308899 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-2fd617a6-2323-4f4d-8b61-df018b572c03 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:37.310138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-2fd617a6-2323-4f4d-8b61-df018b572c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:37.312291 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:37.416224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:37.569411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd7db2-2497-40c8-be48-ccdd2b9ade1c None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:37.570982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-4fdd7db2-2497-40c8-be48-ccdd2b9ade1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:37.724135 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd7db2-2497-40c8-be48-ccdd2b9ade1c None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:37.751044 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:37.838902 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9ad6f-1487-4c70-aa96-418474cbdc00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:37.839548 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9ad6f-1487-4c70-aa96-418474cbdc00 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:37.839935 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9ad6f-1487-4c70-aa96-418474cbdc00 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:37.840211 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9ad6f-1487-4c70-aa96-418474cbdc00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:37.841852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e3b9ad6f-1487-4c70-aa96-418474cbdc00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:37.930761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9ad6f-1487-4c70-aa96-418474cbdc00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:37.932383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9ad6f-1487-4c70-aa96-418474cbdc00 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:37.951587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 711/1440] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:39.969134 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a0cb9-e281-4df6-8e5b-c2b83b0ab6ec None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:39.969741 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a0cb9-e281-4df6-8e5b-c2b83b0ab6ec None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:39.969741 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a0cb9-e281-4df6-8e5b-c2b83b0ab6ec None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:39.971070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:39.977380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc78e8-f85a-4daa-93c8-218ff2d9e308 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:39.977826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc78e8-f85a-4daa-93c8-218ff2d9e308 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:39.977943 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc78e8-f85a-4daa-93c8-218ff2d9e308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:40.024638 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:40.059107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-fbbc78e8-f85a-4daa-93c8-218ff2d9e308 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:40.060591 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-fbbc78e8-f85a-4daa-93c8-218ff2d9e308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:40.175659 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:40.215816 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a0cb9-e281-4df6-8e5b-c2b83b0ab6ec None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:40.218065 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6c4a0cb9-e281-4df6-8e5b-c2b83b0ab6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:40.323158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a0cb9-e281-4df6-8e5b-c2b83b0ab6ec None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:40.358317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:40.405976 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96ed7c-417e-4d4a-99e5-786ada95d780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:40.406629 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96ed7c-417e-4d4a-99e5-786ada95d780 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:40.406912 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96ed7c-417e-4d4a-99e5-786ada95d780 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:40.407186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96ed7c-417e-4d4a-99e5-786ada95d780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:40.408891 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6a96ed7c-417e-4d4a-99e5-786ada95d780 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:40.520910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96ed7c-417e-4d4a-99e5-786ada95d780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:40.522354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96ed7c-417e-4d4a-99e5-786ada95d780 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:40.540949 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 731/1443] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:42.733763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0cbd1-6ce0-43e8-8c82-aa4298227277 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:42.734569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0cbd1-6ce0-43e8-8c82-aa4298227277 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:42.734569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0cbd1-6ce0-43e8-8c82-aa4298227277 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:42.735927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:42.741518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daefdf-7f70-496f-9f58-a86bf09d8599 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:42.741957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daefdf-7f70-496f-9f58-a86bf09d8599 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:42.742074 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daefdf-7f70-496f-9f58-a86bf09d8599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:42.811952 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-69daefdf-7f70-496f-9f58-a86bf09d8599 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:42.813175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-69daefdf-7f70-496f-9f58-a86bf09d8599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:42.826598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:42.911139 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:42.949350 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0cbd1-6ce0-43e8-8c82-aa4298227277 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:42.951635 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6be0cbd1-6ce0-43e8-8c82-aa4298227277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:43.059168 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0cbd1-6ce0-43e8-8c82-aa4298227277 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:43.072121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:43.134649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3d51c-9354-4c57-9503-14028367275e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:43.135293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3d51c-9354-4c57-9503-14028367275e service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:43.135572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3d51c-9354-4c57-9503-14028367275e service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:43.135874 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3d51c-9354-4c57-9503-14028367275e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:43.137507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-11f3d51c-9354-4c57-9503-14028367275e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:43.247022 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3d51c-9354-4c57-9503-14028367275e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:43.249331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3d51c-9354-4c57-9503-14028367275e service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:43.267262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 714/1446] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:45.672643 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7f808-d72c-419c-bf6c-da43a3b9512d None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:45.673319 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7f808-d72c-419c-bf6c-da43a3b9512d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:45.673319 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7f808-d72c-419c-bf6c-da43a3b9512d None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:45.674521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:45.680204 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b927bf-4110-4d99-8149-67e7f5804c31 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:45.680640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b927bf-4110-4d99-8149-67e7f5804c31 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:45.680753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b927bf-4110-4d99-8149-67e7f5804c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:45.728293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:45.746986 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-e9b927bf-4110-4d99-8149-67e7f5804c31 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:45.749074 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e9b927bf-4110-4d99-8149-67e7f5804c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:45.845425 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:45.885383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7f808-d72c-419c-bf6c-da43a3b9512d None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:45.886916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-3ac7f808-d72c-419c-bf6c-da43a3b9512d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:45.997120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7f808-d72c-419c-bf6c-da43a3b9512d None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:46.037114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:46.071549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071e820-d404-49ae-a8e2-ae91c800bbdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:46.072230 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071e820-d404-49ae-a8e2-ae91c800bbdf service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:46.072518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071e820-d404-49ae-a8e2-ae91c800bbdf service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:46.072797 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071e820-d404-49ae-a8e2-ae91c800bbdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:46.074464 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-0071e820-d404-49ae-a8e2-ae91c800bbdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:46.193520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071e820-d404-49ae-a8e2-ae91c800bbdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:46.194961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071e820-d404-49ae-a8e2-ae91c800bbdf service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:46.213706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 734/1449] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:48.455078 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49cd0d-0a02-4494-995a-e92a5b7a36b6 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:48.455612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49cd0d-0a02-4494-995a-e92a5b7a36b6 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:48.455760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49cd0d-0a02-4494-995a-e92a5b7a36b6 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:48.457135 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:48.462490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07790dae-3474-4b1d-8d44-b9e897fff7bb None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:48.462888 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07790dae-3474-4b1d-8d44-b9e897fff7bb None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:48.463056 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07790dae-3474-4b1d-8d44-b9e897fff7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:48.530200 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-07790dae-3474-4b1d-8d44-b9e897fff7bb None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:48.531409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-07790dae-3474-4b1d-8d44-b9e897fff7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:48.551482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:48.627655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:48.666492 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49cd0d-0a02-4494-995a-e92a5b7a36b6 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:48.668088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-4a49cd0d-0a02-4494-995a-e92a5b7a36b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:48.772181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49cd0d-0a02-4494-995a-e92a5b7a36b6 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:48.791968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:48.846430 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bd348-03b9-4262-a505-ed06a65e44ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:48.847081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bd348-03b9-4262-a505-ed06a65e44ce service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:48.847367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bd348-03b9-4262-a505-ed06a65e44ce service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:48.847644 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bd348-03b9-4262-a505-ed06a65e44ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:48.849344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-0e3bd348-03b9-4262-a505-ed06a65e44ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:48.962877 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bd348-03b9-4262-a505-ed06a65e44ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:48.964255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bd348-03b9-4262-a505-ed06a65e44ce service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:48.983701 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 717/1452] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:51.158706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ed619-8855-444c-8e43-2dffea1c12d5 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:51.162808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ed619-8855-444c-8e43-2dffea1c12d5 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:51.162808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ed619-8855-444c-8e43-2dffea1c12d5 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:51.162808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:51.168428 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4c3cd-bd47-41d2-8912-c93abcb76adb None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:51.168953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4c3cd-bd47-41d2-8912-c93abcb76adb None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:51.169080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4c3cd-bd47-41d2-8912-c93abcb76adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:51.240707 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:51.243679 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-a5d4c3cd-bd47-41d2-8912-c93abcb76adb None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:51.245716 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a5d4c3cd-bd47-41d2-8912-c93abcb76adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:51.347330 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:45:51.387378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ed619-8855-444c-8e43-2dffea1c12d5 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:51.390017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b15ed619-8855-444c-8e43-2dffea1c12d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:51.498886 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ed619-8855-444c-8e43-2dffea1c12d5 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:51.550005 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:51.576405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fbe83-0741-477c-97b0-85e82d41115e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:51.577057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fbe83-0741-477c-97b0-85e82d41115e service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:51.577344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fbe83-0741-477c-97b0-85e82d41115e service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:51.577718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fbe83-0741-477c-97b0-85e82d41115e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:51.580294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-457fbe83-0741-477c-97b0-85e82d41115e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:51.698198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fbe83-0741-477c-97b0-85e82d41115e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:51.699814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fbe83-0741-477c-97b0-85e82d41115e service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:51.719573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 737/1455] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:45:53.814784 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93ce41-4c19-432f-8423-d7a38edb1b12 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:53.815325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93ce41-4c19-432f-8423-d7a38edb1b12 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:53.815325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93ce41-4c19-432f-8423-d7a38edb1b12 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:53.816683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:53.822182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa116888-8645-4db8-9dea-86a55aeb160e None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:53.822873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa116888-8645-4db8-9dea-86a55aeb160e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:53.823131 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa116888-8645-4db8-9dea-86a55aeb160e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:53.889081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-fa116888-8645-4db8-9dea-86a55aeb160e None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:53.890328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fa116888-8645-4db8-9dea-86a55aeb160e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:53.919913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:53.990445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:54.030081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93ce41-4c19-432f-8423-d7a38edb1b12 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:54.031626 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5b93ce41-4c19-432f-8423-d7a38edb1b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:54.138039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93ce41-4c19-432f-8423-d7a38edb1b12 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:54.166811 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:54.213109 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c32ce-ac14-4239-82e0-6a40d76c3fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:54.213751 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c32ce-ac14-4239-82e0-6a40d76c3fda service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:54.214038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c32ce-ac14-4239-82e0-6a40d76c3fda service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:54.214317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c32ce-ac14-4239-82e0-6a40d76c3fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:54.216574 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-670c32ce-ac14-4239-82e0-6a40d76c3fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:54.330848 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c32ce-ac14-4239-82e0-6a40d76c3fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:54.332329 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c32ce-ac14-4239-82e0-6a40d76c3fda service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:54.350619 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 720/1458] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:56.516737 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902c1df-aa83-4237-85f0-d52da61cb12b None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:56.517586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902c1df-aa83-4237-85f0-d52da61cb12b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:56.517751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902c1df-aa83-4237-85f0-d52da61cb12b None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:56.519343 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:45:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:56.524816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234d63b-6965-4172-8582-18187cf111a0 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:56.525305 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234d63b-6965-4172-8582-18187cf111a0 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:56.525428 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234d63b-6965-4172-8582-18187cf111a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:56.592847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-6234d63b-6965-4172-8582-18187cf111a0 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:56.594098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6234d63b-6965-4172-8582-18187cf111a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:56.604172 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:56.697425 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:56.736560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902c1df-aa83-4237-85f0-d52da61cb12b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:56.738113 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b902c1df-aa83-4237-85f0-d52da61cb12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:56.844297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902c1df-aa83-4237-85f0-d52da61cb12b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:56.900650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:56.920914 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26015c80-8f01-47ed-98ca-d60d2001f114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:56.921565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26015c80-8f01-47ed-98ca-d60d2001f114 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:56.921851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26015c80-8f01-47ed-98ca-d60d2001f114 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:56.922133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26015c80-8f01-47ed-98ca-d60d2001f114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:56.923824 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-26015c80-8f01-47ed-98ca-d60d2001f114 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:57.035746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26015c80-8f01-47ed-98ca-d60d2001f114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:57.037113 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26015c80-8f01-47ed-98ca-d60d2001f114 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:57.055432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 740/1461] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:45:59.190544 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c897588-985e-43ec-8637-4839779bc125 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:59.191094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c897588-985e-43ec-8637-4839779bc125 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:59.191094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c897588-985e-43ec-8637-4839779bc125 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:59.192528 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:45:59.198059 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590e012-b005-4936-a246-e8e496fe46e4 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:59.198777 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590e012-b005-4936-a246-e8e496fe46e4 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:59.199016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590e012-b005-4936-a246-e8e496fe46e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:59.266509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-1590e012-b005-4936-a246-e8e496fe46e4 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:45:59.267771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1590e012-b005-4936-a246-e8e496fe46e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:59.306362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:59.383870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:45:59.424504 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c897588-985e-43ec-8637-4839779bc125 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:45:59.426151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6c897588-985e-43ec-8637-4839779bc125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:59.534089 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c897588-985e-43ec-8637-4839779bc125 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:45:59.574359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:45:59.612233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec8f55-b0ff-4179-94ef-4b32dcc8b975 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:45:59.612884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec8f55-b0ff-4179-94ef-4b32dcc8b975 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:45:59.613172 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec8f55-b0ff-4179-94ef-4b32dcc8b975 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:45:59.613441 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec8f55-b0ff-4179-94ef-4b32dcc8b975 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:45:59.615085 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-85ec8f55-b0ff-4179-94ef-4b32dcc8b975 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:45:59.729313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec8f55-b0ff-4179-94ef-4b32dcc8b975 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:45:59.730758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec8f55-b0ff-4179-94ef-4b32dcc8b975 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:45:59.749570 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 723/1464] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:45:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:46:01.879861 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6b2fb-a665-46f6-974c-0b2c7fb0dc65 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:01.880444 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6b2fb-a665-46f6-974c-0b2c7fb0dc65 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:01.880544 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6b2fb-a665-46f6-974c-0b2c7fb0dc65 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:01.881985 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:46:01.887890 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac20f9-1fb0-4240-9074-8095d32aa07a None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:01.888889 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac20f9-1fb0-4240-9074-8095d32aa07a None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:01.889038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac20f9-1fb0-4240-9074-8095d32aa07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:01.958002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c2ac20f9-1fb0-4240-9074-8095d32aa07a None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:46:01.959239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c2ac20f9-1fb0-4240-9074-8095d32aa07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:01.978037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:02.063223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:46:02.102793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6b2fb-a665-46f6-974c-0b2c7fb0dc65 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:46:02.105227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c8c6b2fb-a665-46f6-974c-0b2c7fb0dc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:02.216103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6b2fb-a665-46f6-974c-0b2c7fb0dc65 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:46:02.284836 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:02.291977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca71f84-109d-443c-9b05-a76c968e529a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:46:02.292635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca71f84-109d-443c-9b05-a76c968e529a service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:02.292923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca71f84-109d-443c-9b05-a76c968e529a service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:02.293206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca71f84-109d-443c-9b05-a76c968e529a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:02.295392 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-dca71f84-109d-443c-9b05-a76c968e529a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:02.406140 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca71f84-109d-443c-9b05-a76c968e529a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:46:02.407612 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca71f84-109d-443c-9b05-a76c968e529a service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:46:02.426824 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 743/1467] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:46:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:46:04.542226 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56d2e7-2ac3-40eb-ba11-f240976f8461 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:04.542822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56d2e7-2ac3-40eb-ba11-f240976f8461 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:04.542822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56d2e7-2ac3-40eb-ba11-f240976f8461 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:04.544867 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:46:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:46:04.550891 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516ff49-26e2-4720-ad12-782f7218bd31 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:04.551413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516ff49-26e2-4720-ad12-782f7218bd31 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:04.551512 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516ff49-26e2-4720-ad12-782f7218bd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:04.621014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-6516ff49-26e2-4720-ad12-782f7218bd31 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:46:04.622297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6516ff49-26e2-4720-ad12-782f7218bd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:04.671093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:04.727356 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:46:04.767394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56d2e7-2ac3-40eb-ba11-f240976f8461 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:46:04.769002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-4d56d2e7-2ac3-40eb-ba11-f240976f8461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:04.871291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56d2e7-2ac3-40eb-ba11-f240976f8461 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:46:04.919959 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:04.947099 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe34ec4-538f-46cd-a998-ef9c45bdae82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:46:04.947672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe34ec4-538f-46cd-a998-ef9c45bdae82 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:04.947991 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe34ec4-538f-46cd-a998-ef9c45bdae82 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:04.948265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe34ec4-538f-46cd-a998-ef9c45bdae82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:04.949932 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-efe34ec4-538f-46cd-a998-ef9c45bdae82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:05.066402 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe34ec4-538f-46cd-a998-ef9c45bdae82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:46:05.067880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe34ec4-538f-46cd-a998-ef9c45bdae82 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:46:05.088279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 726/1470] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:46:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:46:14.725568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=22 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:15.085678 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=66 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:25.092604 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fde15d-0323-43ba-acd1-d27f95b2c288 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:25.093191 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fde15d-0323-43ba-acd1-d27f95b2c288 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:25.093191 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fde15d-0323-43ba-acd1-d27f95b2c288 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:25.094555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 745/1471] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:46:25.100116 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0693c-a5bd-44a4-bb60-c06cc30567c4 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:25.100534 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0693c-a5bd-44a4-bb60-c06cc30567c4 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:25.100685 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0693c-a5bd-44a4-bb60-c06cc30567c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:25.170875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-09a0693c-a5bd-44a4-bb60-c06cc30567c4 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:46:25.171984 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-09a0693c-a5bd-44a4-bb60-c06cc30567c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:25.269277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 727/1472] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:46:25.355585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace1a8d-8129-4f9e-ace5-1aa6c7c20b67 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:25.356463 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace1a8d-8129-4f9e-ace5-1aa6c7c20b67 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:25.356827 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace1a8d-8129-4f9e-ace5-1aa6c7c20b67 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:25.358195 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 746/1473] 10.222.0.60 () {54 vars in 966 bytes} [Thu Jul 21 20:46:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:46:25.363537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed74a15-c6c4-4299-80a4-1f6c386943f8 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:25.363874 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed74a15-c6c4-4299-80a4-1f6c386943f8 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:25.364185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed74a15-c6c4-4299-80a4-1f6c386943f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:25.441099 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-eed74a15-c6c4-4299-80a4-1f6c386943f8 None None] MFA Rules not processed for user `1d596de177e24ca5a254bda6644a07da`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:46:25.442324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-eed74a15-c6c4-4299-80a4-1f6c386943f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:25.463480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:25.522106 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 728/1474] 10.222.0.60 () {58 vars in 1063 bytes} [Thu Jul 21 20:46:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:46:25.529611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace1a8d-8129-4f9e-ace5-1aa6c7c20b67 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:46:25.531169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6ace1a8d-8129-4f9e-ace5-1aa6c7c20b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:25.644612 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace1a8d-8129-4f9e-ace5-1aa6c7c20b67 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:46:25.728799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d147f16-b214-4626-a455-7d2eb05c0dec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d596de177e24ca5a254bda6644a07da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:46:25.729449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d147f16-b214-4626-a455-7d2eb05c0dec service glance] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:25.729731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d147f16-b214-4626-a455-7d2eb05c0dec service glance] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:25.730032 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d147f16-b214-4626-a455-7d2eb05c0dec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:25.732556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-7d147f16-b214-4626-a455-7d2eb05c0dec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:25.791194 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:25.848148 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d147f16-b214-4626-a455-7d2eb05c0dec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:46:25.849599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d147f16-b214-4626-a455-7d2eb05c0dec service glance] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:46:25.869055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 747/1475] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:46:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:46:28.045412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9179a-863c-4d8b-937c-65b586a09618 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:28.046009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9179a-863c-4d8b-937c-65b586a09618 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:28.046009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9179a-863c-4d8b-937c-65b586a09618 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:28.047350 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 729/1476] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:46:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:46:28.053106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa157945-d0fb-4d7e-a2af-e8f254668cee None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:28.053628 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa157945-d0fb-4d7e-a2af-e8f254668cee None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:28.053730 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa157945-d0fb-4d7e-a2af-e8f254668cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:28.122250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-fa157945-d0fb-4d7e-a2af-e8f254668cee None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:46:28.123475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fa157945-d0fb-4d7e-a2af-e8f254668cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:28.222223 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 748/1477] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:46:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:46:28.246412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9179a-863c-4d8b-937c-65b586a09618 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:46:28.248428 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-50c9179a-863c-4d8b-937c-65b586a09618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:28.396477 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9179a-863c-4d8b-937c-65b586a09618 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:46:28.457122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:28.504936 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cbf95-561e-4c82-b47a-f82920f79c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:46:28.505603 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cbf95-561e-4c82-b47a-f82920f79c89 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:28.505888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cbf95-561e-4c82-b47a-f82920f79c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:28.506164 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cbf95-561e-4c82-b47a-f82920f79c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:28.506910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0cbf95-561e-4c82-b47a-f82920f79c89 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:46:28.508793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: WARNING py.warnings [None req-7d0cbf95-561e-4c82-b47a-f82920f79c89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:46:28.508793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: warnings.warn(msg) Jul 21 20:46:28.508793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: Jul 21 20:46:28.510266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0cbf95-561e-4c82-b47a-f82920f79c89 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:46:28.531916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 730/1478] 10.222.0.60 () {56 vars in 1108 bytes} [Thu Jul 21 20:46:28 2022] GET /identity/v3/projects => generated 2553 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:46:30.649638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756f164-320a-4d7d-9b44-bceaa9aa2d21 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:30.650197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756f164-320a-4d7d-9b44-bceaa9aa2d21 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:30.650197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756f164-320a-4d7d-9b44-bceaa9aa2d21 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:30.651573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 749/1479] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:46:30.657094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514228d-b743-4af4-b835-2becb7d6c749 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:30.657501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514228d-b743-4af4-b835-2becb7d6c749 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:30.657596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514228d-b743-4af4-b835-2becb7d6c749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:30.722192 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-7514228d-b743-4af4-b835-2becb7d6c749 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:46:30.723421 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7514228d-b743-4af4-b835-2becb7d6c749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:30.823041 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 731/1480] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:46:30.872039 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756f164-320a-4d7d-9b44-bceaa9aa2d21 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:46:30.873613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9756f164-320a-4d7d-9b44-bceaa9aa2d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:30.883388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:31.038595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756f164-320a-4d7d-9b44-bceaa9aa2d21 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:46:31.128041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:31.162687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbdccb-d7e5-4740-b28a-4148aa4683b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:46:31.163458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbdccb-d7e5-4740-b28a-4148aa4683b7 service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:31.163786 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbdccb-d7e5-4740-b28a-4148aa4683b7 service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:31.164073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbdccb-d7e5-4740-b28a-4148aa4683b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:31.165770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-60bbdccb-d7e5-4740-b28a-4148aa4683b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:31.259813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbdccb-d7e5-4740-b28a-4148aa4683b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:46:31.261445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbdccb-d7e5-4740-b28a-4148aa4683b7 service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:46:31.280275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 750/1481] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:46:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:46:33.588185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1b55a-af22-45a3-8c81-b2a4dce0b7e7 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:33.588691 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1b55a-af22-45a3-8c81-b2a4dce0b7e7 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:33.588814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1b55a-af22-45a3-8c81-b2a4dce0b7e7 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:33.590136 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 732/1482] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:46:33.595457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a9ef0-0404-4c18-9c68-67ef937d91fa None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:33.595938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a9ef0-0404-4c18-9c68-67ef937d91fa None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:33.596037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a9ef0-0404-4c18-9c68-67ef937d91fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:33.661585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-601a9ef0-0404-4c18-9c68-67ef937d91fa None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:46:33.662736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-601a9ef0-0404-4c18-9c68-67ef937d91fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:33.709665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:33.761414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 751/1483] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:46:33.798651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1b55a-af22-45a3-8c81-b2a4dce0b7e7 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:46:33.800259 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-50c1b55a-af22-45a3-8c81-b2a4dce0b7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:33.824102 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:33.906373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1b55a-af22-45a3-8c81-b2a4dce0b7e7 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:46:33.980180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ac18f-996a-4cf5-95db-88dfd51d0ce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:46:33.981346 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ac18f-996a-4cf5-95db-88dfd51d0ce7 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:33.981681 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ac18f-996a-4cf5-95db-88dfd51d0ce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:33.981963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ac18f-996a-4cf5-95db-88dfd51d0ce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:33.983619 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ed0ac18f-996a-4cf5-95db-88dfd51d0ce7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:34.060611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:34.095871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ac18f-996a-4cf5-95db-88dfd51d0ce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:46:34.097310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ac18f-996a-4cf5-95db-88dfd51d0ce7 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:46:34.119384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 733/1484] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:46:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:46:36.242186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afab6d-2b49-47af-8418-462e53501da0 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:36.242765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afab6d-2b49-47af-8418-462e53501da0 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:36.242765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afab6d-2b49-47af-8418-462e53501da0 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:36.244077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 752/1485] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:46:36.250023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052f62b-0c37-49fc-8f5f-2797ad460e71 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:36.251764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052f62b-0c37-49fc-8f5f-2797ad460e71 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:36.251764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052f62b-0c37-49fc-8f5f-2797ad460e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:36.317640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-6052f62b-0c37-49fc-8f5f-2797ad460e71 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:46:36.318871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6052f62b-0c37-49fc-8f5f-2797ad460e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:36.417026 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 734/1486] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:46:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:46:36.456526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afab6d-2b49-47af-8418-462e53501da0 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:46:36.458073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a4afab6d-2b49-47af-8418-462e53501da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:36.691518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afab6d-2b49-47af-8418-462e53501da0 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:46:36.765636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:36.767210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0e239-d387-44a2-ad48-1484b84977f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:46:36.767954 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0e239-d387-44a2-ad48-1484b84977f1 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:36.768235 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0e239-d387-44a2-ad48-1484b84977f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:36.768515 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0e239-d387-44a2-ad48-1484b84977f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:36.770805 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d6e0e239-d387-44a2-ad48-1484b84977f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:36.889901 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0e239-d387-44a2-ad48-1484b84977f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:46:36.891346 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0e239-d387-44a2-ad48-1484b84977f1 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:46:36.910000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 753/1487] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:46:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:46:39.141913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea0ef9-e8c7-4476-ae8f-2590e969976c None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:39.142490 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea0ef9-e8c7-4476-ae8f-2590e969976c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:39.142490 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea0ef9-e8c7-4476-ae8f-2590e969976c None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:39.143883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 735/1488] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:46:39.149869 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ceddb-85c0-4b60-ac0f-51a75c2d8ec3 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:39.150336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ceddb-85c0-4b60-ac0f-51a75c2d8ec3 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:39.150442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ceddb-85c0-4b60-ac0f-51a75c2d8ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:39.224549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-126ceddb-85c0-4b60-ac0f-51a75c2d8ec3 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:46:39.225788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-126ceddb-85c0-4b60-ac0f-51a75c2d8ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:39.279593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:39.327860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 754/1489] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:46:39.375999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea0ef9-e8c7-4476-ae8f-2590e969976c None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:46:39.377576 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b8ea0ef9-e8c7-4476-ae8f-2590e969976c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:39.423362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:39.506037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea0ef9-e8c7-4476-ae8f-2590e969976c None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:46:39.591063 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45642427-3dd7-4703-bdb2-767ad4ef5726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:46:39.591741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45642427-3dd7-4703-bdb2-767ad4ef5726 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:39.592037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45642427-3dd7-4703-bdb2-767ad4ef5726 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:39.592311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45642427-3dd7-4703-bdb2-767ad4ef5726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:39.594340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-45642427-3dd7-4703-bdb2-767ad4ef5726 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:39.683703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:39.715671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45642427-3dd7-4703-bdb2-767ad4ef5726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:46:39.718102 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45642427-3dd7-4703-bdb2-767ad4ef5726 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:46:39.738270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 736/1490] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:46:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:46:42.075043 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cec69-1640-4bb5-9be7-2f6e8985948b None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:42.075621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cec69-1640-4bb5-9be7-2f6e8985948b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:42.075779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cec69-1640-4bb5-9be7-2f6e8985948b None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:42.077141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 755/1491] 10.222.0.60 () {54 vars in 926 bytes} [Thu Jul 21 20:46:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:46:42.082761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c382b8f-012c-4921-a34f-d6fd7dbaa9cf None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:42.083214 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c382b8f-012c-4921-a34f-d6fd7dbaa9cf None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:42.083318 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c382b8f-012c-4921-a34f-d6fd7dbaa9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:42.152403 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-7c382b8f-012c-4921-a34f-d6fd7dbaa9cf None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:46:42.153766 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7c382b8f-012c-4921-a34f-d6fd7dbaa9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:42.256897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 737/1492] 10.222.0.60 () {58 vars in 1023 bytes} [Thu Jul 21 20:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:46:42.298523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500cec69-1640-4bb5-9be7-2f6e8985948b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:46:42.300401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-500cec69-1640-4bb5-9be7-2f6e8985948b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:42.410018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500cec69-1640-4bb5-9be7-2f6e8985948b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:46:42.486547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fd3c7-cee1-4f7c-afb8-551c5ccb67b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:46:42.487230 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fd3c7-cee1-4f7c-afb8-551c5ccb67b5 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:46:42.487510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fd3c7-cee1-4f7c-afb8-551c5ccb67b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:46:42.487816 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fd3c7-cee1-4f7c-afb8-551c5ccb67b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:46:42.489488 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c38fd3c7-cee1-4f7c-afb8-551c5ccb67b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:46:42.498976 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:42.604492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38fd3c7-cee1-4f7c-afb8-551c5ccb67b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:46:42.605971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38fd3c7-cee1-4f7c-afb8-551c5ccb67b5 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:46:42.625035 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 756/1493] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:46:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:46:52.254958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=82 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:46:52.622449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=50 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:48:11.199725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00025170-37f2-4c85-b73f-f5176bb1294c None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:11.200337 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00025170-37f2-4c85-b73f-f5176bb1294c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:11.200337 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00025170-37f2-4c85-b73f-f5176bb1294c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:11.280996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-00025170-37f2-4c85-b73f-f5176bb1294c None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:48:11.282247 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-00025170-37f2-4c85-b73f-f5176bb1294c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:11.378594 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 738/1494] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:48:11.386684 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc265c-dbd8-4fd2-8666-90cc312e2e07 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:11.387217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc265c-dbd8-4fd2-8666-90cc312e2e07 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:11.387329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc265c-dbd8-4fd2-8666-90cc312e2e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:11.460655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-74bc265c-dbd8-4fd2-8666-90cc312e2e07 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:48:11.461773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-74bc265c-dbd8-4fd2-8666-90cc312e2e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:11.556727 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 757/1495] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:48:11.561471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00025170-37f2-4c85-b73f-f5176bb1294c None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:48:11.562672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-00025170-37f2-4c85-b73f-f5176bb1294c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:11.574024 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00025170-37f2-4c85-b73f-f5176bb1294c None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:48:11.636262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:48:11.676932 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a082b-9672-42db-aa4d-5c67584f4460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:48:11.677615 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a082b-9672-42db-aa4d-5c67584f4460 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:11.677902 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a082b-9672-42db-aa4d-5c67584f4460 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:11.678179 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a082b-9672-42db-aa4d-5c67584f4460 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:11.678918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a082b-9672-42db-aa4d-5c67584f4460 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:48:11.679287 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a082b-9672-42db-aa4d-5c67584f4460 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:48:11.680379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: WARNING py.warnings [None req-7b0a082b-9672-42db-aa4d-5c67584f4460 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:48:11.680379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: warnings.warn(msg) Jul 21 20:48:11.680379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: Jul 21 20:48:11.681307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a082b-9672-42db-aa4d-5c67584f4460 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:48:11.701833 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 739/1496] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:48:11.702988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc265c-dbd8-4fd2-8666-90cc312e2e07 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:48:11.713621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc265c-dbd8-4fd2-8666-90cc312e2e07 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:48:11.776289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:48:11.811666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4a119-287e-42b2-b73a-46ccdbbe56f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:48:11.812344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4a119-287e-42b2-b73a-46ccdbbe56f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:11.812625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4a119-287e-42b2-b73a-46ccdbbe56f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:11.812900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4a119-287e-42b2-b73a-46ccdbbe56f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:11.813592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4a119-287e-42b2-b73a-46ccdbbe56f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:48:11.815358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: WARNING py.warnings [None req-03b4a119-287e-42b2-b73a-46ccdbbe56f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:48:11.815358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: warnings.warn(msg) Jul 21 20:48:11.815358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: Jul 21 20:48:11.816756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4a119-287e-42b2-b73a-46ccdbbe56f4 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:48:11.872533 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 758/1497] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:48:11 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:48:11.876877 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a082b-9672-42db-aa4d-5c67584f4460 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:48:11.878395 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7b0a082b-9672-42db-aa4d-5c67584f4460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:11.913576 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a082b-9672-42db-aa4d-5c67584f4460 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:48:12.006912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72427d6a-c319-4694-a98d-16dafcc9b917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:48:12.007495 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72427d6a-c319-4694-a98d-16dafcc9b917 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:12.007804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72427d6a-c319-4694-a98d-16dafcc9b917 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:12.008079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72427d6a-c319-4694-a98d-16dafcc9b917 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:12.008760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72427d6a-c319-4694-a98d-16dafcc9b917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:48:12.010263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: WARNING py.warnings [None req-72427d6a-c319-4694-a98d-16dafcc9b917 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:48:12.010263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: warnings.warn(msg) Jul 21 20:48:12.010263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: Jul 21 20:48:12.011463 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72427d6a-c319-4694-a98d-16dafcc9b917 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:48:12.053918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 740/1498] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:48:11 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:48:12.057902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4a119-287e-42b2-b73a-46ccdbbe56f4 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:48:12.059567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-03b4a119-287e-42b2-b73a-46ccdbbe56f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:12.122037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4a119-287e-42b2-b73a-46ccdbbe56f4 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:48:12.205993 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:48:12.217966 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac78bd-369a-4bca-a040-1cc3dc30dfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:48:12.218633 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac78bd-369a-4bca-a040-1cc3dc30dfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:12.218926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac78bd-369a-4bca-a040-1cc3dc30dfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:12.219203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac78bd-369a-4bca-a040-1cc3dc30dfb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:12.219981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac78bd-369a-4bca-a040-1cc3dc30dfb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:48:12.220992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: WARNING py.warnings [None req-78ac78bd-369a-4bca-a040-1cc3dc30dfb8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:48:12.220992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: warnings.warn(msg) Jul 21 20:48:12.220992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: Jul 21 20:48:12.221875 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac78bd-369a-4bca-a040-1cc3dc30dfb8 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:48:12.235134 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 759/1499] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:48:12.239239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72427d6a-c319-4694-a98d-16dafcc9b917 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:48:12.240789 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-72427d6a-c319-4694-a98d-16dafcc9b917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:12.252224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72427d6a-c319-4694-a98d-16dafcc9b917 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:48:12.277306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:48:12.345781 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc2ad8-d716-4659-8b9a-5c84c5a87905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:48:12.346495 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc2ad8-d716-4659-8b9a-5c84c5a87905 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:12.346777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc2ad8-d716-4659-8b9a-5c84c5a87905 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:12.347050 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc2ad8-d716-4659-8b9a-5c84c5a87905 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:12.347804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc2ad8-d716-4659-8b9a-5c84c5a87905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:48:12.348827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: WARNING py.warnings [None req-9cfc2ad8-d716-4659-8b9a-5c84c5a87905 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:48:12.348827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: warnings.warn(msg) Jul 21 20:48:12.348827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: Jul 21 20:48:12.349704 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc2ad8-d716-4659-8b9a-5c84c5a87905 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:48:12.363139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 741/1500] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:48:12.367373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac78bd-369a-4bca-a040-1cc3dc30dfb8 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:48:12.369123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-78ac78bd-369a-4bca-a040-1cc3dc30dfb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:12.401805 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac78bd-369a-4bca-a040-1cc3dc30dfb8 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:48:12.496354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2e1ee-588f-4a58-9eec-d0b4831b9592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:48:12.497259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2e1ee-588f-4a58-9eec-d0b4831b9592 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:12.497523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2e1ee-588f-4a58-9eec-d0b4831b9592 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:12.497798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2e1ee-588f-4a58-9eec-d0b4831b9592 admin admin] PATH_INFO: `/v3/projects/4a27595b6f0040c68dd401ae241afd64/users/a5f892cbb4494d0aa9be926055327e61/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:12.519863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2e1ee-588f-4a58-9eec-d0b4831b9592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a27595b6f0040c68dd401ae241afd64, user_id=a5f892cbb4494d0aa9be926055327e61, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:48:12.520893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: WARNING py.warnings [None req-cfa2e1ee-588f-4a58-9eec-d0b4831b9592 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:48:12.520893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: warnings.warn(msg) Jul 21 20:48:12.520893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: Jul 21 20:48:12.522847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2e1ee-588f-4a58-9eec-d0b4831b9592 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:48:12.547968 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 760/1501] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:48:12 2022] PUT /identity/v3/projects/4a27595b6f0040c68dd401ae241afd64/users/a5f892cbb4494d0aa9be926055327e61/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:48:12.560014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7f9f9-74d4-4ecd-bdee-63e8f2a7d132 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:12.560304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7f9f9-74d4-4ecd-bdee-63e8f2a7d132 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:12.560588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7f9f9-74d4-4ecd-bdee-63e8f2a7d132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:12.616099 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-7eb7f9f9-74d4-4ecd-bdee-63e8f2a7d132 None None] MFA Rules not processed for user `a5f892cbb4494d0aa9be926055327e61`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:48:12.617314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7eb7f9f9-74d4-4ecd-bdee-63e8f2a7d132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:12.695314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 742/1502] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:48:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:48:12.709096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae83dc-eb0b-4e91-a8bf-1f2eeb90929e None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:12.709393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae83dc-eb0b-4e91-a8bf-1f2eeb90929e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:12.709679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae83dc-eb0b-4e91-a8bf-1f2eeb90929e None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:12.711049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 761/1503] 10.222.0.60 () {54 vars in 964 bytes} [Thu Jul 21 20:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:48:12.716442 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce8671-34d4-489d-bfcb-6e12bd63753f None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:12.716736 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce8671-34d4-489d-bfcb-6e12bd63753f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:12.717017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce8671-34d4-489d-bfcb-6e12bd63753f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:12.746486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:48:12.792109 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-cfce8671-34d4-489d-bfcb-6e12bd63753f None None] MFA Rules not processed for user `a8eef7746fc4417e8d4de0b052f3eaa0`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:48:12.793317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-cfce8671-34d4-489d-bfcb-6e12bd63753f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:12.875210 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 743/1504] 10.222.0.60 () {58 vars in 1061 bytes} [Thu Jul 21 20:48:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:48:12.882813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae83dc-eb0b-4e91-a8bf-1f2eeb90929e None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:48:12.884415 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-75ae83dc-eb0b-4e91-a8bf-1f2eeb90929e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:12.939796 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:48:12.994578 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae83dc-eb0b-4e91-a8bf-1f2eeb90929e None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:48:13.069367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3bbfd3-e0e9-4725-863d-385e2c1b76ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:48:13.070050 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bbfd3-e0e9-4725-863d-385e2c1b76ac service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:13.070333 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bbfd3-e0e9-4725-863d-385e2c1b76ac service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:13.070606 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bbfd3-e0e9-4725-863d-385e2c1b76ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:13.072426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-5b3bbfd3-e0e9-4725-863d-385e2c1b76ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:13.175362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3bbfd3-e0e9-4725-863d-385e2c1b76ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:48:13.176733 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3bbfd3-e0e9-4725-863d-385e2c1b76ac service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:48:13.180193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:48:13.195368 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 762/1505] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:48:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:48:13.238729 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17545185-03da-4230-a30d-34648d186f07 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:13.239016 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17545185-03da-4230-a30d-34648d186f07 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:13.239291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17545185-03da-4230-a30d-34648d186f07 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:13.240612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 744/1506] 10.222.0.60 () {52 vars in 829 bytes} [Thu Jul 21 20:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:48:13.244702 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3bbfd3-e0e9-4725-863d-385e2c1b76ac service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:48:13.245928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-5b3bbfd3-e0e9-4725-863d-385e2c1b76ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:13.347407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3bbfd3-e0e9-4725-863d-385e2c1b76ac service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:48:13.440787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f3a2d-0f89-4a8a-8294-a0d5f970e3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:48:13.441562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f3a2d-0f89-4a8a-8294-a0d5f970e3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:13.441846 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f3a2d-0f89-4a8a-8294-a0d5f970e3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:13.442123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f3a2d-0f89-4a8a-8294-a0d5f970e3cd admin admin] PATH_INFO: `/v3/users/a5f892cbb4494d0aa9be926055327e61` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:13.454852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f3a2d-0f89-4a8a-8294-a0d5f970e3cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5f892cbb4494d0aa9be926055327e61)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:48:13.455919 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: WARNING py.warnings [None req-701f3a2d-0f89-4a8a-8294-a0d5f970e3cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:48:13.455919 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: warnings.warn(msg) Jul 21 20:48:13.455919 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: Jul 21 20:48:13.457135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f3a2d-0f89-4a8a-8294-a0d5f970e3cd admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:48:13.467288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:48:13.528642 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-701f3a2d-0f89-4a8a-8294-a0d5f970e3cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5f892cbb4494d0aa9be926055327e61'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:48:13.529085 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-701f3a2d-0f89-4a8a-8294-a0d5f970e3cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5f892cbb4494d0aa9be926055327e61'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:48:13.547321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-701f3a2d-0f89-4a8a-8294-a0d5f970e3cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5f892cbb4494d0aa9be926055327e61'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:48:13.556773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-701f3a2d-0f89-4a8a-8294-a0d5f970e3cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5f892cbb4494d0aa9be926055327e61'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:48:13.571222 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-701f3a2d-0f89-4a8a-8294-a0d5f970e3cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5f892cbb4494d0aa9be926055327e61'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:48:13.577840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 763/1507] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:48:13 2022] DELETE /identity/v3/users/a5f892cbb4494d0aa9be926055327e61 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:48:13.582348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce890b-6daf-446c-806e-9bf8626c059f None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:13.582637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce890b-6daf-446c-806e-9bf8626c059f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:13.582937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce890b-6daf-446c-806e-9bf8626c059f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:13.642256 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-85ce890b-6daf-446c-806e-9bf8626c059f None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:48:13.643497 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-85ce890b-6daf-446c-806e-9bf8626c059f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:13.844476 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:48:13.858507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 745/1508] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:48:13.870264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f3a2d-0f89-4a8a-8294-a0d5f970e3cd admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:48:13.871476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-701f3a2d-0f89-4a8a-8294-a0d5f970e3cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:13.971053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f3a2d-0f89-4a8a-8294-a0d5f970e3cd admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:48:14.029103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:48:14.051361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1a3d4-ea0b-4a52-9950-7428bd00b6dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:48:14.052077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1a3d4-ea0b-4a52-9950-7428bd00b6dc service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:14.052339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1a3d4-ea0b-4a52-9950-7428bd00b6dc service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:14.052617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1a3d4-ea0b-4a52-9950-7428bd00b6dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:14.054321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-29d1a3d4-ea0b-4a52-9950-7428bd00b6dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:14.179397 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1a3d4-ea0b-4a52-9950-7428bd00b6dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:48:14.180778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1a3d4-ea0b-4a52-9950-7428bd00b6dc service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:48:14.200327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 764/1509] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:48:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:48:14.536851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce890b-6daf-446c-806e-9bf8626c059f None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:48:14.538138 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-85ce890b-6daf-446c-806e-9bf8626c059f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:48:14.551248 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce890b-6daf-446c-806e-9bf8626c059f None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:48:14.656058 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c2c66-69d2-46a6-8d35-71fbb707a7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:48:14.656924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c2c66-69d2-46a6-8d35-71fbb707a7f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:48:14.657209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c2c66-69d2-46a6-8d35-71fbb707a7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:48:14.657486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c2c66-69d2-46a6-8d35-71fbb707a7f9 admin admin] PATH_INFO: `/v3/projects/4a27595b6f0040c68dd401ae241afd64` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:48:14.667843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c2c66-69d2-46a6-8d35-71fbb707a7f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a27595b6f0040c68dd401ae241afd64)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:48:14.668872 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: WARNING py.warnings [None req-499c2c66-69d2-46a6-8d35-71fbb707a7f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:48:14.668872 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: warnings.warn(msg) Jul 21 20:48:14.668872 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: Jul 21 20:48:14.670078 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c2c66-69d2-46a6-8d35-71fbb707a7f9 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:48:14.726687 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-499c2c66-69d2-46a6-8d35-71fbb707a7f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a27595b6f0040c68dd401ae241afd64', 'request_id': 'req-499c2c66-69d2-46a6-8d35-71fbb707a7f9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:48:14.737414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-499c2c66-69d2-46a6-8d35-71fbb707a7f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a27595b6f0040c68dd401ae241afd64', 'request_id': 'req-499c2c66-69d2-46a6-8d35-71fbb707a7f9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:48:14.743648 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-499c2c66-69d2-46a6-8d35-71fbb707a7f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a27595b6f0040c68dd401ae241afd64', 'request_id': 'req-499c2c66-69d2-46a6-8d35-71fbb707a7f9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:48:14.774312 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-499c2c66-69d2-46a6-8d35-71fbb707a7f9 admin admin] The token cache is being invalidate because project 4a27595b6f0040c68dd401ae241afd64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:48:14.774603 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-499c2c66-69d2-46a6-8d35-71fbb707a7f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:48:14.775069 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-499c2c66-69d2-46a6-8d35-71fbb707a7f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:48:14.776629 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 746/1510] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:48:14 2022] DELETE /identity/v3/projects/4a27595b6f0040c68dd401ae241afd64 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:48:24.197883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=62 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:48:24.772342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:50:03.483083 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab30ca-67eb-40b8-9d8b-bd9e762386e8 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:03.483703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab30ca-67eb-40b8-9d8b-bd9e762386e8 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:03.483703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab30ca-67eb-40b8-9d8b-bd9e762386e8 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:03.486190 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:50:03.492856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25fe03-42cf-479a-8217-a9c8e0a7c5a8 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:03.493360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25fe03-42cf-479a-8217-a9c8e0a7c5a8 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:03.493464 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25fe03-42cf-479a-8217-a9c8e0a7c5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:03.588578 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-de25fe03-42cf-479a-8217-a9c8e0a7c5a8 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:50:03.591114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-de25fe03-42cf-479a-8217-a9c8e0a7c5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:03.711628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:50:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:50:03.878057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab30ca-67eb-40b8-9d8b-bd9e762386e8 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:50:03.880380 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-3fab30ca-67eb-40b8-9d8b-bd9e762386e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:04.016877 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab30ca-67eb-40b8-9d8b-bd9e762386e8 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:50:04.022760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43696c-8665-44ad-b2f8-68677fa2d99e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:50:04.023697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43696c-8665-44ad-b2f8-68677fa2d99e service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:04.023951 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43696c-8665-44ad-b2f8-68677fa2d99e service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:04.024048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43696c-8665-44ad-b2f8-68677fa2d99e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:04.034888 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43696c-8665-44ad-b2f8-68677fa2d99e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:50:04.036812 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43696c-8665-44ad-b2f8-68677fa2d99e service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:50:04.048511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 766/1513] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:50:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:50:04.923035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f0c14-b3cc-48b9-a8f1-2458c635629a None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:04.923611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f0c14-b3cc-48b9-a8f1-2458c635629a None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:04.923611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f0c14-b3cc-48b9-a8f1-2458c635629a None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:04.925054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:50:04.930323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d138b3-c022-4782-89a9-7e442400d9f0 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:04.930815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d138b3-c022-4782-89a9-7e442400d9f0 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:04.930909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d138b3-c022-4782-89a9-7e442400d9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:04.950361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e1d138b3-c022-4782-89a9-7e442400d9f0 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:50:04.951630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e1d138b3-c022-4782-89a9-7e442400d9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:04.959789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:50:05.113110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f0c14-b3cc-48b9-a8f1-2458c635629a None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:50:05.128759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f0c14-b3cc-48b9-a8f1-2458c635629a None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:50:05.135114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d33395-44f1-49f0-8901-be4e900472f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:50:05.137014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d33395-44f1-49f0-8901-be4e900472f3 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:05.137478 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d33395-44f1-49f0-8901-be4e900472f3 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:05.137769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d33395-44f1-49f0-8901-be4e900472f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:05.146712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d33395-44f1-49f0-8901-be4e900472f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:50:05.148302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d33395-44f1-49f0-8901-be4e900472f3 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:50:05.159695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 749/1516] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:50:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:50:07.641759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a0edc-85cb-4b21-8f2a-46a665bb7a3f None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:07.642337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a0edc-85cb-4b21-8f2a-46a665bb7a3f None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:07.642337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a0edc-85cb-4b21-8f2a-46a665bb7a3f None None] PATH_INFO: `/v3` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:07.643869 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 768/1517] 10.222.0.73 () {54 vars in 925 bytes} [Thu Jul 21 20:50:07 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:50:07.653911 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71789c-537f-4a04-9070-23fe2bed1d7f None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:07.654433 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71789c-537f-4a04-9070-23fe2bed1d7f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:07.654433 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71789c-537f-4a04-9070-23fe2bed1d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:07.726469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-cf71789c-537f-4a04-9070-23fe2bed1d7f None None] MFA Rules not processed for user `f4786051321f4a6f80a6cd23eacb09bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:50:07.727689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-cf71789c-537f-4a04-9070-23fe2bed1d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:07.749830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:50:07.807409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 750/1518] 10.222.0.73 () {58 vars in 1015 bytes} [Thu Jul 21 20:50:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:50:14.945225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=56 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:50:16.400248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1ada9-aac3-476b-a968-8cdb2f48aee3 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:16.400899 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1ada9-aac3-476b-a968-8cdb2f48aee3 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:16.401060 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1ada9-aac3-476b-a968-8cdb2f48aee3 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:16.403570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 769/1519] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:50:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:50:16.409794 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c37ac3-3f02-43d6-b720-183c12086384 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:16.410265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c37ac3-3f02-43d6-b720-183c12086384 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:16.410371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c37ac3-3f02-43d6-b720-183c12086384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:16.429699 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-43c37ac3-3f02-43d6-b720-183c12086384 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:50:16.430947 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-43c37ac3-3f02-43d6-b720-183c12086384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:16.442494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 751/1520] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:50:16.609895 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1ada9-aac3-476b-a968-8cdb2f48aee3 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:50:16.625613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1ada9-aac3-476b-a968-8cdb2f48aee3 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:50:16.631764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390480b-9763-43d9-8cd2-2d2195d81d9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:50:16.632414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390480b-9763-43d9-8cd2-2d2195d81d9f service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:16.632619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390480b-9763-43d9-8cd2-2d2195d81d9f service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:16.632900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390480b-9763-43d9-8cd2-2d2195d81d9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:16.642795 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390480b-9763-43d9-8cd2-2d2195d81d9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:50:16.644550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390480b-9763-43d9-8cd2-2d2195d81d9f service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:50:16.655261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 770/1521] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:50:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:50:17.500132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edb588-8a09-4dc3-8112-7f6b8c17c948 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:17.500706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edb588-8a09-4dc3-8112-7f6b8c17c948 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:17.500706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edb588-8a09-4dc3-8112-7f6b8c17c948 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:17.502086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:50:17.507443 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41c5ae-26eb-4e77-bfc8-5f6df109419d None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:17.507926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41c5ae-26eb-4e77-bfc8-5f6df109419d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:17.508021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41c5ae-26eb-4e77-bfc8-5f6df109419d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:17.526973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-2b41c5ae-26eb-4e77-bfc8-5f6df109419d None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:50:17.533321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-2b41c5ae-26eb-4e77-bfc8-5f6df109419d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:17.544823 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:50:17.701820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edb588-8a09-4dc3-8112-7f6b8c17c948 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:50:17.717290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edb588-8a09-4dc3-8112-7f6b8c17c948 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:50:17.723176 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f51dc-d366-4900-8888-2cbe9ea7cf75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:50:17.723901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f51dc-d366-4900-8888-2cbe9ea7cf75 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:17.724171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f51dc-d366-4900-8888-2cbe9ea7cf75 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:17.724446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f51dc-d366-4900-8888-2cbe9ea7cf75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:17.735331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f51dc-d366-4900-8888-2cbe9ea7cf75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:50:17.737817 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f51dc-d366-4900-8888-2cbe9ea7cf75 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:50:17.752875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 753/1524] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:50:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:50:19.769503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d797c-da86-4c8a-9a36-209d0c8b2827 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:19.770040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d797c-da86-4c8a-9a36-209d0c8b2827 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:19.770040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d797c-da86-4c8a-9a36-209d0c8b2827 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:19.771341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 772/1525] 10.222.0.60 () {54 vars in 961 bytes} [Thu Jul 21 20:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:50:19.777598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f5013-7836-49fb-a376-49ed0e873815 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:19.778132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f5013-7836-49fb-a376-49ed0e873815 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:19.778228 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f5013-7836-49fb-a376-49ed0e873815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:19.799466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-f02f5013-7836-49fb-a376-49ed0e873815 None None] MFA Rules not processed for user `f4786051321f4a6f80a6cd23eacb09bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:50:19.800740 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f02f5013-7836-49fb-a376-49ed0e873815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:19.809597 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 754/1526] 10.222.0.60 () {58 vars in 1058 bytes} [Thu Jul 21 20:50:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:50:19.817340 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d797c-da86-4c8a-9a36-209d0c8b2827 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:50:19.831097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d797c-da86-4c8a-9a36-209d0c8b2827 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:50:19.836475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc24d17-9f5f-45af-abfb-ef0f5c863f2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:50:19.837129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc24d17-9f5f-45af-abfb-ef0f5c863f2b service placement] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:19.837418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc24d17-9f5f-45af-abfb-ef0f5c863f2b service placement] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:19.837703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc24d17-9f5f-45af-abfb-ef0f5c863f2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:19.847532 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc24d17-9f5f-45af-abfb-ef0f5c863f2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:50:19.849172 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc24d17-9f5f-45af-abfb-ef0f5c863f2b service placement] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:50:19.860927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 773/1527] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:50:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:50:28.979798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb88784-9020-43de-b786-97bb3ce6dbb7 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:28.980478 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb88784-9020-43de-b786-97bb3ce6dbb7 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:28.980478 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb88784-9020-43de-b786-97bb3ce6dbb7 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:28.981668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:50:28.987079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebc2a1-512d-4145-bc06-b881f2638f6a None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:28.987567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebc2a1-512d-4145-bc06-b881f2638f6a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:28.987665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebc2a1-512d-4145-bc06-b881f2638f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:29.007794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e3ebc2a1-512d-4145-bc06-b881f2638f6a None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:50:29.009019 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e3ebc2a1-512d-4145-bc06-b881f2638f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:29.017707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:50:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:50:29.183846 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb88784-9020-43de-b786-97bb3ce6dbb7 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:50:29.195060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb88784-9020-43de-b786-97bb3ce6dbb7 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:50:29.201589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e8b02-74ca-45d0-95e1-37260ccce151 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:50:29.202253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e8b02-74ca-45d0-95e1-37260ccce151 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:29.202536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e8b02-74ca-45d0-95e1-37260ccce151 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:29.202813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e8b02-74ca-45d0-95e1-37260ccce151 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:29.211743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e8b02-74ca-45d0-95e1-37260ccce151 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:50:29.213336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e8b02-74ca-45d0-95e1-37260ccce151 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:50:29.224548 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 756/1530] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:50:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:50:30.052784 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a222b0-f8f7-45d3-929d-97e02cdade86 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:30.053357 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a222b0-f8f7-45d3-929d-97e02cdade86 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:30.053357 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a222b0-f8f7-45d3-929d-97e02cdade86 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:30.054701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 20:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:50:30.059839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650aad2-aa58-4801-9f52-a10860324f99 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:30.060363 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650aad2-aa58-4801-9f52-a10860324f99 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:30.060474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650aad2-aa58-4801-9f52-a10860324f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:30.079437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-0650aad2-aa58-4801-9f52-a10860324f99 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:50:30.080683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-0650aad2-aa58-4801-9f52-a10860324f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:30.088572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 20:50:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:50:30.244414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a222b0-f8f7-45d3-929d-97e02cdade86 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:50:30.260581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a222b0-f8f7-45d3-929d-97e02cdade86 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:50:30.266844 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a8e48-c33e-4d0b-b3bf-e33ad99f984b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:50:30.267577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a8e48-c33e-4d0b-b3bf-e33ad99f984b service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:30.267893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a8e48-c33e-4d0b-b3bf-e33ad99f984b service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:30.268171 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a8e48-c33e-4d0b-b3bf-e33ad99f984b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:30.277111 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a8e48-c33e-4d0b-b3bf-e33ad99f984b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:50:30.279142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a8e48-c33e-4d0b-b3bf-e33ad99f984b service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:50:30.289486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 776/1533] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:50:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:50:34.466950 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c0d73-8ae6-4b01-9c8c-fabd93b0f8bb None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:34.467593 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c0d73-8ae6-4b01-9c8c-fabd93b0f8bb None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:34.467593 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c0d73-8ae6-4b01-9c8c-fabd93b0f8bb None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:34.469003 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:50:34.474804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcdff4-986f-4dc0-b05a-8a78d61134be None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:34.475261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcdff4-986f-4dc0-b05a-8a78d61134be None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:34.475355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcdff4-986f-4dc0-b05a-8a78d61134be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:34.496383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-a6fcdff4-986f-4dc0-b05a-8a78d61134be None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:50:34.497574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a6fcdff4-986f-4dc0-b05a-8a78d61134be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:34.506160 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:50:34.585608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c0d73-8ae6-4b01-9c8c-fabd93b0f8bb None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:50:34.587432 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-020c0d73-8ae6-4b01-9c8c-fabd93b0f8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:34.686849 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c0d73-8ae6-4b01-9c8c-fabd93b0f8bb None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:50:34.690463 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfdc2f-5f6c-42ff-93cb-99c85392bb12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:50:34.691110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfdc2f-5f6c-42ff-93cb-99c85392bb12 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:34.691389 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfdc2f-5f6c-42ff-93cb-99c85392bb12 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:34.691662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfdc2f-5f6c-42ff-93cb-99c85392bb12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:34.701968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfdc2f-5f6c-42ff-93cb-99c85392bb12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:50:34.703412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfdc2f-5f6c-42ff-93cb-99c85392bb12 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:50:34.713314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 759/1536] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:50:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:50:36.800817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800ad47-9b78-4af7-818c-344faff60d08 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:36.801387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800ad47-9b78-4af7-818c-344faff60d08 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:36.801387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800ad47-9b78-4af7-818c-344faff60d08 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:36.802912 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:50:36.809229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50ae56-1cdf-430b-8f51-e28e64cff485 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:36.809721 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50ae56-1cdf-430b-8f51-e28e64cff485 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:36.809845 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50ae56-1cdf-430b-8f51-e28e64cff485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:36.827793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-da50ae56-1cdf-430b-8f51-e28e64cff485 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:50:36.829026 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-da50ae56-1cdf-430b-8f51-e28e64cff485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:36.837104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:50:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:50:36.878025 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800ad47-9b78-4af7-818c-344faff60d08 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:50:36.879822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-5800ad47-9b78-4af7-818c-344faff60d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:36.893329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800ad47-9b78-4af7-818c-344faff60d08 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:50:36.896963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d3530-78a6-4369-91f7-642ab51f647f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:50:36.897638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d3530-78a6-4369-91f7-642ab51f647f service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:36.897890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d3530-78a6-4369-91f7-642ab51f647f service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:36.898168 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d3530-78a6-4369-91f7-642ab51f647f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:36.908164 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d3530-78a6-4369-91f7-642ab51f647f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:50:36.909600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d3530-78a6-4369-91f7-642ab51f647f service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:50:36.920170 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 779/1539] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:50:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:50:40.826342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033024b-f5f7-4e11-8437-09fd65093edc None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:40.826954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033024b-f5f7-4e11-8437-09fd65093edc None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:40.826954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033024b-f5f7-4e11-8437-09fd65093edc None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:40.828402 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 761/1540] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:50:40.833844 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68588b46-8969-4603-938a-e156707dfd94 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:40.834288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68588b46-8969-4603-938a-e156707dfd94 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:40.834407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68588b46-8969-4603-938a-e156707dfd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:40.852846 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-68588b46-8969-4603-938a-e156707dfd94 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:50:40.854035 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-68588b46-8969-4603-938a-e156707dfd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:40.862782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 780/1541] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:50:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:50:41.170151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033024b-f5f7-4e11-8437-09fd65093edc None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:50:41.171917 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e033024b-f5f7-4e11-8437-09fd65093edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:41.176064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=83 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:50:41.262064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033024b-f5f7-4e11-8437-09fd65093edc None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:50:41.266343 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a13bca-b59b-4271-af47-80c6c78b081a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:50:41.266998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a13bca-b59b-4271-af47-80c6c78b081a service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:41.267281 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a13bca-b59b-4271-af47-80c6c78b081a service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:41.267563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a13bca-b59b-4271-af47-80c6c78b081a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:41.278540 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a13bca-b59b-4271-af47-80c6c78b081a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:50:41.280097 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a13bca-b59b-4271-af47-80c6c78b081a service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:50:41.290190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 762/1542] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:50:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:50:43.420339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37999bb-68dd-4e16-88c4-5d9d9a296e8d None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:43.421299 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37999bb-68dd-4e16-88c4-5d9d9a296e8d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:43.421580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37999bb-68dd-4e16-88c4-5d9d9a296e8d None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:43.423666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 781/1543] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 20:50:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:50:43.430153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76f27b-03b6-488e-bd94-d8c6108f238e None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:43.430693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76f27b-03b6-488e-bd94-d8c6108f238e None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:43.430693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76f27b-03b6-488e-bd94-d8c6108f238e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:43.450592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-da76f27b-03b6-488e-bd94-d8c6108f238e None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:50:43.451843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-da76f27b-03b6-488e-bd94-d8c6108f238e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:43.462202 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 763/1544] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 20:50:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:50:43.773711 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37999bb-68dd-4e16-88c4-5d9d9a296e8d None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:50:43.775453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e37999bb-68dd-4e16-88c4-5d9d9a296e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:50:43.788540 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37999bb-68dd-4e16-88c4-5d9d9a296e8d None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:50:43.792476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bea03-4bb5-4d8d-a193-8d52cc33399b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:50:43.793126 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bea03-4bb5-4d8d-a193-8d52cc33399b service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:50:43.793411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bea03-4bb5-4d8d-a193-8d52cc33399b service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:50:43.793687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bea03-4bb5-4d8d-a193-8d52cc33399b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:50:43.803010 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bea03-4bb5-4d8d-a193-8d52cc33399b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:50:43.804475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bea03-4bb5-4d8d-a193-8d52cc33399b service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:50:43.815155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 782/1545] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:50:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:50:53.445624 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=37 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:50:53.810660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=42 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:22.551435 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d5a23-4b48-4fb9-93b8-4fb56262dd26 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:22.552528 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d5a23-4b48-4fb9-93b8-4fb56262dd26 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:22.553049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d5a23-4b48-4fb9-93b8-4fb56262dd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:22.574481 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-a10d5a23-4b48-4fb9-93b8-4fb56262dd26 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:22.576139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a10d5a23-4b48-4fb9-93b8-4fb56262dd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:22.592004 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 764/1546] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:22.600720 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1597008-8d9b-4977-906e-8211fbc0025c None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:22.601355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198324c-41d1-4320-a25f-8094abf51646 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:22.601355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198324c-41d1-4320-a25f-8094abf51646 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:22.601355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198324c-41d1-4320-a25f-8094abf51646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:22.602299 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1597008-8d9b-4977-906e-8211fbc0025c None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:22.602808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1597008-8d9b-4977-906e-8211fbc0025c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:22.633991 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-9198324c-41d1-4320-a25f-8094abf51646 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:22.636154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-9198324c-41d1-4320-a25f-8094abf51646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:22.648768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d1597008-8d9b-4977-906e-8211fbc0025c None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:22.650817 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 765/1547] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:52:22.651612 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d1597008-8d9b-4977-906e-8211fbc0025c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:22.655278 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ee52b-e8e1-41e8-b24e-a59b4d2b4517 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:22.655818 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ee52b-e8e1-41e8-b24e-a59b4d2b4517 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:22.656369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ee52b-e8e1-41e8-b24e-a59b4d2b4517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:22.671619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 783/1548] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:52:22.672851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1597008-8d9b-4977-906e-8211fbc0025c None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:22.686357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-576ee52b-e8e1-41e8-b24e-a59b4d2b4517 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:22.687408 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1597008-8d9b-4977-906e-8211fbc0025c None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:22.688098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-576ee52b-e8e1-41e8-b24e-a59b4d2b4517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:22.698717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d65a9-5ef3-43eb-8e95-ca57d0105d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:22.703704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d65a9-5ef3-43eb-8e95-ca57d0105d25 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:22.703704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d65a9-5ef3-43eb-8e95-ca57d0105d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:22.703704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d65a9-5ef3-43eb-8e95-ca57d0105d25 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:22.703704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d65a9-5ef3-43eb-8e95-ca57d0105d25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:52:22.703704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d65a9-5ef3-43eb-8e95-ca57d0105d25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:22.704609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 766/1549] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:52:22.705248 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca1f37-bf30-4562-b13a-96cb8d0d7508 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:22.705538 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca1f37-bf30-4562-b13a-96cb8d0d7508 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:22.705817 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca1f37-bf30-4562-b13a-96cb8d0d7508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:22.708014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: WARNING py.warnings [None req-634d65a9-5ef3-43eb-8e95-ca57d0105d25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:52:22.708014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: warnings.warn(msg) Jul 21 20:52:22.708014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: Jul 21 20:52:22.708921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d65a9-5ef3-43eb-8e95-ca57d0105d25 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:22.731568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 784/1550] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:52:22.732956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97523b-7d17-43c0-97a8-6925dc81208d None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:22.733250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97523b-7d17-43c0-97a8-6925dc81208d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:22.733531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97523b-7d17-43c0-97a8-6925dc81208d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:22.743755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-3bca1f37-bf30-4562-b13a-96cb8d0d7508 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:22.745022 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-3bca1f37-bf30-4562-b13a-96cb8d0d7508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:22.767303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 767/1551] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:22.770047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a4730-ac41-4b48-b0af-4b3c353185cc None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:22.770337 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a4730-ac41-4b48-b0af-4b3c353185cc None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:22.770614 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a4730-ac41-4b48-b0af-4b3c353185cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:22.781674 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-5e97523b-7d17-43c0-97a8-6925dc81208d None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:22.785450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-5e97523b-7d17-43c0-97a8-6925dc81208d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:22.812492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 785/1552] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:22.813645 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97523b-7d17-43c0-97a8-6925dc81208d None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:22.815755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-f10a4730-ac41-4b48-b0af-4b3c353185cc None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:22.816109 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f10a4730-ac41-4b48-b0af-4b3c353185cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:22.831282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 768/1553] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:22.832415 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97523b-7d17-43c0-97a8-6925dc81208d None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:22.832630 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a8309-800f-47f3-b605-eb04ca24c575 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:22.832928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a8309-800f-47f3-b605-eb04ca24c575 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:22.833208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a8309-800f-47f3-b605-eb04ca24c575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:22.849741 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e1da1-a62a-4e68-b761-b10f753ce9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:22.850795 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e1da1-a62a-4e68-b761-b10f753ce9ff admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:22.851079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e1da1-a62a-4e68-b761-b10f753ce9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:22.851352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e1da1-a62a-4e68-b761-b10f753ce9ff admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:22.852239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e1da1-a62a-4e68-b761-b10f753ce9ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:22.854927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e1da1-a62a-4e68-b761-b10f753ce9ff admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:22.860693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c22a8309-800f-47f3-b605-eb04ca24c575 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:22.862179 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c22a8309-800f-47f3-b605-eb04ca24c575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:22.871551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 769/1554] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:22.872459 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a8309-800f-47f3-b605-eb04ca24c575 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:22.885443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a8309-800f-47f3-b605-eb04ca24c575 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:22.890888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c558c71-4659-4cb0-9db5-13d5554b1301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:22.891579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c558c71-4659-4cb0-9db5-13d5554b1301 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:22.891909 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c558c71-4659-4cb0-9db5-13d5554b1301 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:22.892184 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c558c71-4659-4cb0-9db5-13d5554b1301 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:22.892941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c558c71-4659-4cb0-9db5-13d5554b1301 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:52:22.893315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c558c71-4659-4cb0-9db5-13d5554b1301 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:22.894847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c558c71-4659-4cb0-9db5-13d5554b1301 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:22.908182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 786/1555] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:22 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:22.909850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 770/1556] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:52:22.910267 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e1da1-a62a-4e68-b761-b10f753ce9ff admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:22.911410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99f682-96a4-47f2-a3bb-8878c17c8235 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:22.911752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99f682-96a4-47f2-a3bb-8878c17c8235 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:22.912032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99f682-96a4-47f2-a3bb-8878c17c8235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:22.937587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-ec99f682-96a4-47f2-a3bb-8878c17c8235 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:22.939262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ec99f682-96a4-47f2-a3bb-8878c17c8235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:22.995768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e1da1-a62a-4e68-b761-b10f753ce9ff admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.001524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbae598-1339-45c2-a84e-0634a0125515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.002661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbae598-1339-45c2-a84e-0634a0125515 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.003174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbae598-1339-45c2-a84e-0634a0125515 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.003669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbae598-1339-45c2-a84e-0634a0125515 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.005104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbae598-1339-45c2-a84e-0634a0125515 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:52:23.005629 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbae598-1339-45c2-a84e-0634a0125515 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.007346 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbae598-1339-45c2-a84e-0634a0125515 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.012613 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 771/1557] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:23.016468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf6c63-3389-4bfc-bea4-f7de480f71e7 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.016721 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf6c63-3389-4bfc-bea4-f7de480f71e7 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.017003 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf6c63-3389-4bfc-bea4-f7de480f71e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.030189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 787/1558] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:52:23.030189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbae598-1339-45c2-a84e-0634a0125515 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.035548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbae598-1339-45c2-a84e-0634a0125515 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.039831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c6cf6c63-3389-4bfc-bea4-f7de480f71e7 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:23.040258 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f0f69-b884-4117-855b-2cba86a0327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.040923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f0f69-b884-4117-855b-2cba86a0327e admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.041070 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c6cf6c63-3389-4bfc-bea4-f7de480f71e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:23.041498 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f0f69-b884-4117-855b-2cba86a0327e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.042461 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f0f69-b884-4117-855b-2cba86a0327e admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.043519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f0f69-b884-4117-855b-2cba86a0327e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:52:23.043863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f0f69-b884-4117-855b-2cba86a0327e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.045461 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f0f69-b884-4117-855b-2cba86a0327e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.049810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 772/1559] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:23.050619 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf6c63-3389-4bfc-bea4-f7de480f71e7 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.059754 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 788/1560] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:52:23.060434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f0f69-b884-4117-855b-2cba86a0327e admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.064795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf6c63-3389-4bfc-bea4-f7de480f71e7 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.070442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f0f69-b884-4117-855b-2cba86a0327e admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.075753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9b560-16e3-4133-8a6b-83782a28c9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.076423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9b560-16e3-4133-8a6b-83782a28c9f1 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.076725 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9b560-16e3-4133-8a6b-83782a28c9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.077028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9b560-16e3-4133-8a6b-83782a28c9f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.077733 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9b560-16e3-4133-8a6b-83782a28c9f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.077847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40daf52-afe5-41a3-9c6c-f6cb07b74bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.078483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40daf52-afe5-41a3-9c6c-f6cb07b74bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.078772 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40daf52-afe5-41a3-9c6c-f6cb07b74bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.079052 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40daf52-afe5-41a3-9c6c-f6cb07b74bc5 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.079642 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9b560-16e3-4133-8a6b-83782a28c9f1 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.079776 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40daf52-afe5-41a3-9c6c-f6cb07b74bc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.081707 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40daf52-afe5-41a3-9c6c-f6cb07b74bc5 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.105493 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 773/1561] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/users => generated 283 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:23.106809 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873b70b-8362-46d4-afe5-b1a6d91294d8 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.107112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 789/1562] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:23.110381 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873b70b-8362-46d4-afe5-b1a6d91294d8 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.110381 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873b70b-8362-46d4-afe5-b1a6d91294d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.110548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da9869-24d6-49b2-95a5-15cba0babad8 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.110548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da9869-24d6-49b2-95a5-15cba0babad8 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.110548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da9869-24d6-49b2-95a5-15cba0babad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.127937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-2873b70b-8362-46d4-afe5-b1a6d91294d8 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:23.129177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-2873b70b-8362-46d4-afe5-b1a6d91294d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:23.130759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d8da9869-24d6-49b2-95a5-15cba0babad8 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:23.131983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d8da9869-24d6-49b2-95a5-15cba0babad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:23.193457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 790/1563] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:23.197275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51aee0-6171-4407-a829-f2a1541b84cc None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.198107 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51aee0-6171-4407-a829-f2a1541b84cc None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.198393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51aee0-6171-4407-a829-f2a1541b84cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.203335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 774/1564] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:23.204221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873b70b-8362-46d4-afe5-b1a6d91294d8 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.206216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:23.215633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873b70b-8362-46d4-afe5-b1a6d91294d8 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.220378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f2e35-85d0-4542-8c94-0a2f11c07a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.221050 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f2e35-85d0-4542-8c94-0a2f11c07a93 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.221333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f2e35-85d0-4542-8c94-0a2f11c07a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.221641 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f2e35-85d0-4542-8c94-0a2f11c07a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.222213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-ee51aee0-6171-4407-a829-f2a1541b84cc None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:23.222361 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f2e35-85d0-4542-8c94-0a2f11c07a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.223353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: WARNING py.warnings [None req-d68f2e35-85d0-4542-8c94-0a2f11c07a93 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:52:23.223353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: warnings.warn(msg) Jul 21 20:52:23.223353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: Jul 21 20:52:23.223626 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ee51aee0-6171-4407-a829-f2a1541b84cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:23.224635 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f2e35-85d0-4542-8c94-0a2f11c07a93 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.231859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 791/1565] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:23.233439 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635a344-eb1e-4810-8504-fc08f51fd0d5 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.233439 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635a344-eb1e-4810-8504-fc08f51fd0d5 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.233691 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635a344-eb1e-4810-8504-fc08f51fd0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.252336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 775/1566] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:23.253051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-3635a344-eb1e-4810-8504-fc08f51fd0d5 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:23.253630 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f2e35-85d0-4542-8c94-0a2f11c07a93 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.254383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-3635a344-eb1e-4810-8504-fc08f51fd0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:23.258321 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:23.321866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f2e35-85d0-4542-8c94-0a2f11c07a93 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.326593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 792/1567] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:23.326593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635a344-eb1e-4810-8504-fc08f51fd0d5 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.328856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438c1dd-efcc-4282-bd67-0b408558fbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.329648 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438c1dd-efcc-4282-bd67-0b408558fbe0 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.329934 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438c1dd-efcc-4282-bd67-0b408558fbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.330211 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438c1dd-efcc-4282-bd67-0b408558fbe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.330914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438c1dd-efcc-4282-bd67-0b408558fbe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.332951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438c1dd-efcc-4282-bd67-0b408558fbe0 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.334413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635a344-eb1e-4810-8504-fc08f51fd0d5 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.339796 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4aca8-81ed-4c85-b726-842d31e6b246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.340475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4aca8-81ed-4c85-b726-842d31e6b246 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.340756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4aca8-81ed-4c85-b726-842d31e6b246 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.341032 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4aca8-81ed-4c85-b726-842d31e6b246 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.341774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4aca8-81ed-4c85-b726-842d31e6b246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.349643 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4aca8-81ed-4c85-b726-842d31e6b246 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.354012 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 793/1568] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:23.355051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4aca8-81ed-4c85-b726-842d31e6b246 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.365509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4aca8-81ed-4c85-b726-842d31e6b246 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.371359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adf255-568f-4fed-9939-5dfdb2934bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.372155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adf255-568f-4fed-9939-5dfdb2934bc3 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.372502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adf255-568f-4fed-9939-5dfdb2934bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.372853 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adf255-568f-4fed-9939-5dfdb2934bc3 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.373812 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adf255-568f-4fed-9939-5dfdb2934bc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.374935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: WARNING py.warnings [None req-14adf255-568f-4fed-9939-5dfdb2934bc3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:52:23.374935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: warnings.warn(msg) Jul 21 20:52:23.374935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: Jul 21 20:52:23.380674 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 776/1569] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/projects => generated 393 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:23.380936 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adf255-568f-4fed-9939-5dfdb2934bc3 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.381122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e262026-cd61-4a4b-9ecb-52139250dab6 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.381122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e262026-cd61-4a4b-9ecb-52139250dab6 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.381122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e262026-cd61-4a4b-9ecb-52139250dab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.398814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-8e262026-cd61-4a4b-9ecb-52139250dab6 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:23.400546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8e262026-cd61-4a4b-9ecb-52139250dab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:23.408778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 794/1570] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/users => generated 287 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:23.409901 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793d966-64b5-4bb5-a823-f7c42a1cefd8 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.410259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793d966-64b5-4bb5-a823-f7c42a1cefd8 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.410608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793d966-64b5-4bb5-a823-f7c42a1cefd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.443627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-a793d966-64b5-4bb5-a823-f7c42a1cefd8 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:23.445191 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a793d966-64b5-4bb5-a823-f7c42a1cefd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:23.478853 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 777/1571] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:23.479704 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e262026-cd61-4a4b-9ecb-52139250dab6 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.491892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e262026-cd61-4a4b-9ecb-52139250dab6 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.497161 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76646592-03d6-4806-be51-6b97af365719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.497825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76646592-03d6-4806-be51-6b97af365719 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.498110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76646592-03d6-4806-be51-6b97af365719 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.498387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76646592-03d6-4806-be51-6b97af365719 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.499126 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76646592-03d6-4806-be51-6b97af365719 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:52:23.499435 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76646592-03d6-4806-be51-6b97af365719 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.501075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76646592-03d6-4806-be51-6b97af365719 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.510919 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 795/1572] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:23.511765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793d966-64b5-4bb5-a823-f7c42a1cefd8 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.520550 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 778/1573] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:52:23.520550 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76646592-03d6-4806-be51-6b97af365719 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.524748 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76646592-03d6-4806-be51-6b97af365719 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.525815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793d966-64b5-4bb5-a823-f7c42a1cefd8 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.530083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087a8fb-ea2a-47b4-ba18-376fd2cdf21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.530392 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd3b04-8292-49b8-89bc-b0dbb2ed1f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.530901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087a8fb-ea2a-47b4-ba18-376fd2cdf21f admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.531064 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd3b04-8292-49b8-89bc-b0dbb2ed1f8a admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.531173 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087a8fb-ea2a-47b4-ba18-376fd2cdf21f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.531349 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd3b04-8292-49b8-89bc-b0dbb2ed1f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.531446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087a8fb-ea2a-47b4-ba18-376fd2cdf21f admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.531630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd3b04-8292-49b8-89bc-b0dbb2ed1f8a admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.532189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087a8fb-ea2a-47b4-ba18-376fd2cdf21f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:52:23.532364 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd3b04-8292-49b8-89bc-b0dbb2ed1f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.532550 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087a8fb-ea2a-47b4-ba18-376fd2cdf21f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.534286 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087a8fb-ea2a-47b4-ba18-376fd2cdf21f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.534404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd3b04-8292-49b8-89bc-b0dbb2ed1f8a admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.548821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 779/1574] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:52:23.549730 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087a8fb-ea2a-47b4-ba18-376fd2cdf21f admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.557574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 796/1575] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:23.558525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd3b04-8292-49b8-89bc-b0dbb2ed1f8a admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.563499 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087a8fb-ea2a-47b4-ba18-376fd2cdf21f admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.568281 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56349d4-8bd9-4259-ad1c-6e3e4f610f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.568909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd3b04-8292-49b8-89bc-b0dbb2ed1f8a admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.569079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56349d4-8bd9-4259-ad1c-6e3e4f610f60 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.569225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56349d4-8bd9-4259-ad1c-6e3e4f610f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.569500 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56349d4-8bd9-4259-ad1c-6e3e4f610f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.570231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56349d4-8bd9-4259-ad1c-6e3e4f610f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.571854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56349d4-8bd9-4259-ad1c-6e3e4f610f60 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.574296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ef699-6b15-4b7d-8d3f-5e0edd2f57a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.575082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ef699-6b15-4b7d-8d3f-5e0edd2f57a1 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.575361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ef699-6b15-4b7d-8d3f-5e0edd2f57a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.575632 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ef699-6b15-4b7d-8d3f-5e0edd2f57a1 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.576362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ef699-6b15-4b7d-8d3f-5e0edd2f57a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.578266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ef699-6b15-4b7d-8d3f-5e0edd2f57a1 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.582986 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 780/1576] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:23.583870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56349d4-8bd9-4259-ad1c-6e3e4f610f60 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.598683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56349d4-8bd9-4259-ad1c-6e3e4f610f60 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.606543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 797/1577] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/users => generated 297 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:23.606543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ef699-6b15-4b7d-8d3f-5e0edd2f57a1 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.607225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a7589-c7eb-43b7-94dd-e2f94698a043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.607225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a7589-c7eb-43b7-94dd-e2f94698a043 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.607225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a7589-c7eb-43b7-94dd-e2f94698a043 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.607225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a7589-c7eb-43b7-94dd-e2f94698a043 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.607560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a7589-c7eb-43b7-94dd-e2f94698a043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.607830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a7589-c7eb-43b7-94dd-e2f94698a043 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.614453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ef699-6b15-4b7d-8d3f-5e0edd2f57a1 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.621785 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 781/1578] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:23.622636 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a7589-c7eb-43b7-94dd-e2f94698a043 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.626216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34efa9-0512-4edc-a2a0-3e14c051f2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.627175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34efa9-0512-4edc-a2a0-3e14c051f2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.627316 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34efa9-0512-4edc-a2a0-3e14c051f2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.627556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34efa9-0512-4edc-a2a0-3e14c051f2e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.628794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34efa9-0512-4edc-a2a0-3e14c051f2e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:52:23.629110 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34efa9-0512-4edc-a2a0-3e14c051f2e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.631875 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34efa9-0512-4edc-a2a0-3e14c051f2e6 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.634125 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a7589-c7eb-43b7-94dd-e2f94698a043 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.639605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec619b6c-3260-4393-9f06-eae902fc133c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.640295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec619b6c-3260-4393-9f06-eae902fc133c admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.640573 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec619b6c-3260-4393-9f06-eae902fc133c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.640845 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec619b6c-3260-4393-9f06-eae902fc133c admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.641570 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec619b6c-3260-4393-9f06-eae902fc133c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:52:23.641871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec619b6c-3260-4393-9f06-eae902fc133c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.643463 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec619b6c-3260-4393-9f06-eae902fc133c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.652389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 798/1579] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:52:23.652389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34efa9-0512-4edc-a2a0-3e14c051f2e6 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.656619 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 782/1580] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:52:23.657439 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec619b6c-3260-4393-9f06-eae902fc133c admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.666756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34efa9-0512-4edc-a2a0-3e14c051f2e6 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.668201 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec619b6c-3260-4393-9f06-eae902fc133c admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.671776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b72ee-2974-46f3-be3d-0267162a0cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.672446 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b72ee-2974-46f3-be3d-0267162a0cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.672728 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b72ee-2974-46f3-be3d-0267162a0cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.672852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287cc51-3d00-4282-a7ca-2045dab008c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.673006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b72ee-2974-46f3-be3d-0267162a0cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.673417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287cc51-3d00-4282-a7ca-2045dab008c8 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.673697 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287cc51-3d00-4282-a7ca-2045dab008c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.673968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287cc51-3d00-4282-a7ca-2045dab008c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.674355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b72ee-2974-46f3-be3d-0267162a0cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.674655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287cc51-3d00-4282-a7ca-2045dab008c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.676302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b72ee-2974-46f3-be3d-0267162a0cc8 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.676568 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287cc51-3d00-4282-a7ca-2045dab008c8 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.698080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:23.714094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 783/1582] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:23.714094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287cc51-3d00-4282-a7ca-2045dab008c8 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.714622 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 799/1582] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/projects => generated 405 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:23.715911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b72ee-2974-46f3-be3d-0267162a0cc8 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.716312 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:23.780192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b72ee-2974-46f3-be3d-0267162a0cc8 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.792188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287cc51-3d00-4282-a7ca-2045dab008c8 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.797487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367ad07-ea71-4287-92a8-590e687990bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.798170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367ad07-ea71-4287-92a8-590e687990bf admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.798457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367ad07-ea71-4287-92a8-590e687990bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.798739 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367ad07-ea71-4287-92a8-590e687990bf admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.799477 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367ad07-ea71-4287-92a8-590e687990bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.801155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367ad07-ea71-4287-92a8-590e687990bf admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.812217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 784/1583] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:23.813140 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367ad07-ea71-4287-92a8-590e687990bf admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.823778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367ad07-ea71-4287-92a8-590e687990bf admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.828625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de77379-095e-461a-b29b-125836b955f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.829309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de77379-095e-461a-b29b-125836b955f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.829592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de77379-095e-461a-b29b-125836b955f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.829867 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de77379-095e-461a-b29b-125836b955f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.830596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de77379-095e-461a-b29b-125836b955f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.832248 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de77379-095e-461a-b29b-125836b955f7 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.843216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 785/1584] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:23.844089 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de77379-095e-461a-b29b-125836b955f7 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.850372 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bb422-9dad-48ae-8bca-6f6bd7bcaee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.851671 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bb422-9dad-48ae-8bca-6f6bd7bcaee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.851820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bb422-9dad-48ae-8bca-6f6bd7bcaee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.852070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bb422-9dad-48ae-8bca-6f6bd7bcaee3 admin admin] PATH_INFO: `/v3/projects/511c4e2782a54c32b4f856a9cda6710a/users/06ab1297aa0d4fd1aa345280e4be7e1c/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.856716 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de77379-095e-461a-b29b-125836b955f7 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.861729 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458806b7-8855-4e68-a72d-54799ac351b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.862395 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458806b7-8855-4e68-a72d-54799ac351b0 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.862679 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458806b7-8855-4e68-a72d-54799ac351b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.862957 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458806b7-8855-4e68-a72d-54799ac351b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.863693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458806b7-8855-4e68-a72d-54799ac351b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.865326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458806b7-8855-4e68-a72d-54799ac351b0 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.868222 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bb422-9dad-48ae-8bca-6f6bd7bcaee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=511c4e2782a54c32b4f856a9cda6710a, user_id=06ab1297aa0d4fd1aa345280e4be7e1c, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.871567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bb422-9dad-48ae-8bca-6f6bd7bcaee3 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.876457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 786/1585] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:23.877594 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458806b7-8855-4e68-a72d-54799ac351b0 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.887625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458806b7-8855-4e68-a72d-54799ac351b0 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.888419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 800/1586] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:23 2022] PUT /identity/v3/projects/511c4e2782a54c32b4f856a9cda6710a/users/06ab1297aa0d4fd1aa345280e4be7e1c/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:23.889368 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bb422-9dad-48ae-8bca-6f6bd7bcaee3 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:23.952070 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4418c01-1102-4fa2-b210-b53bb896849e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.952675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4418c01-1102-4fa2-b210-b53bb896849e admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.953085 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4418c01-1102-4fa2-b210-b53bb896849e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.953360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4418c01-1102-4fa2-b210-b53bb896849e admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.954053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4418c01-1102-4fa2-b210-b53bb896849e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.956016 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4418c01-1102-4fa2-b210-b53bb896849e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.960383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bb422-9dad-48ae-8bca-6f6bd7bcaee3 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:23.965573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9dcc7-9b2c-493f-b9d8-7a75ef5625b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:23.966239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9dcc7-9b2c-493f-b9d8-7a75ef5625b1 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:23.966523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9dcc7-9b2c-493f-b9d8-7a75ef5625b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:23.966801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9dcc7-9b2c-493f-b9d8-7a75ef5625b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:23.967499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9dcc7-9b2c-493f-b9d8-7a75ef5625b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:23.969468 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9dcc7-9b2c-493f-b9d8-7a75ef5625b1 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:23.987821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 787/1587] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:23.987821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4418c01-1102-4fa2-b210-b53bb896849e admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.007446 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 801/1588] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:24.007446 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9dcc7-9b2c-493f-b9d8-7a75ef5625b1 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.022592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:24.056800 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4418c01-1102-4fa2-b210-b53bb896849e admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.066364 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:24.081079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9dcc7-9b2c-493f-b9d8-7a75ef5625b1 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.086344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c9a8a-4665-4a34-b993-b93131360b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.087060 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c9a8a-4665-4a34-b993-b93131360b65 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.087342 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c9a8a-4665-4a34-b993-b93131360b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.087618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c9a8a-4665-4a34-b993-b93131360b65 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.088483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c9a8a-4665-4a34-b993-b93131360b65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.090490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c9a8a-4665-4a34-b993-b93131360b65 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.116490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 802/1589] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:24 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:24.116490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c9a8a-4665-4a34-b993-b93131360b65 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.121683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d650f59-c984-48b0-bc66-a0d54b59b1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.122372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650f59-c984-48b0-bc66-a0d54b59b1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.122712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650f59-c984-48b0-bc66-a0d54b59b1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.122995 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650f59-c984-48b0-bc66-a0d54b59b1d5 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.123702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d650f59-c984-48b0-bc66-a0d54b59b1d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.125494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c9a8a-4665-4a34-b993-b93131360b65 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.125714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d650f59-c984-48b0-bc66-a0d54b59b1d5 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.131207 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb8ace-403e-4ee8-8974-6492905f4ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.132092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb8ace-403e-4ee8-8974-6492905f4ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.132493 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb8ace-403e-4ee8-8974-6492905f4ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.132842 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb8ace-403e-4ee8-8974-6492905f4ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.133646 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb8ace-403e-4ee8-8974-6492905f4ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.135778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb8ace-403e-4ee8-8974-6492905f4ef5 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.147093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 803/1590] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:24.148147 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb8ace-403e-4ee8-8974-6492905f4ef5 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.151458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 788/1591] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:23 2022] POST /identity/v3/users => generated 303 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:24.151458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d650f59-c984-48b0-bc66-a0d54b59b1d5 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.160294 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d650f59-c984-48b0-bc66-a0d54b59b1d5 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.161286 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb8ace-403e-4ee8-8974-6492905f4ef5 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.165101 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96124c4b-d1a8-40ca-9500-6a2692f6f9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.165960 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c102099-e6b7-40ec-a643-a5ea64a01308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.166122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96124c4b-d1a8-40ca-9500-6a2692f6f9a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.166289 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96124c4b-d1a8-40ca-9500-6a2692f6f9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.167108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96124c4b-d1a8-40ca-9500-6a2692f6f9a6 admin admin] PATH_INFO: `/v3/projects/4b69684bfd6e47f2b6e5bb801b3140d8/users/82764d7145a24ccc841052a776d1e4a1/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.167226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c102099-e6b7-40ec-a643-a5ea64a01308 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.167445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c102099-e6b7-40ec-a643-a5ea64a01308 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.167761 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c102099-e6b7-40ec-a643-a5ea64a01308 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.168476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c102099-e6b7-40ec-a643-a5ea64a01308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.170077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c102099-e6b7-40ec-a643-a5ea64a01308 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.181519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 804/1592] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:24.183409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a6a03-c4cf-4169-90d8-d80715505ab3 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.183701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a6a03-c4cf-4169-90d8-d80715505ab3 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.184004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a6a03-c4cf-4169-90d8-d80715505ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.187065 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96124c4b-d1a8-40ca-9500-6a2692f6f9a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b69684bfd6e47f2b6e5bb801b3140d8, user_id=82764d7145a24ccc841052a776d1e4a1, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.188142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: WARNING py.warnings [None req-96124c4b-d1a8-40ca-9500-6a2692f6f9a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:52:24.188142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: warnings.warn(msg) Jul 21 20:52:24.188142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: Jul 21 20:52:24.190091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96124c4b-d1a8-40ca-9500-6a2692f6f9a6 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.204714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 789/1593] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:24 2022] PUT /identity/v3/projects/4b69684bfd6e47f2b6e5bb801b3140d8/users/82764d7145a24ccc841052a776d1e4a1/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:24.205660 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96124c4b-d1a8-40ca-9500-6a2692f6f9a6 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.220283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-b05a6a03-c4cf-4169-90d8-d80715505ab3 None None] MFA Rules not processed for user `06ab1297aa0d4fd1aa345280e4be7e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:24.221531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b05a6a03-c4cf-4169-90d8-d80715505ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:24.274271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96124c4b-d1a8-40ca-9500-6a2692f6f9a6 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.279606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a503a-d3cb-4af3-9a71-a86f5d4d4101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.280361 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a503a-d3cb-4af3-9a71-a86f5d4d4101 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.280779 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a503a-d3cb-4af3-9a71-a86f5d4d4101 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.280958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a503a-d3cb-4af3-9a71-a86f5d4d4101 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.281655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a503a-d3cb-4af3-9a71-a86f5d4d4101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.283552 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a503a-d3cb-4af3-9a71-a86f5d4d4101 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.294631 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 805/1594] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:24.294631 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a6a03-c4cf-4169-90d8-d80715505ab3 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.303741 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a6a03-c4cf-4169-90d8-d80715505ab3 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.308572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530f610-f3a4-463b-86b2-213eeee87b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.312226 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 790/1595] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:24 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:24.312389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530f610-f3a4-463b-86b2-213eeee87b8b admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.312389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530f610-f3a4-463b-86b2-213eeee87b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.312389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530f610-f3a4-463b-86b2-213eeee87b8b admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.312668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a503a-d3cb-4af3-9a71-a86f5d4d4101 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.312772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530f610-f3a4-463b-86b2-213eeee87b8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.312772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530f610-f3a4-463b-86b2-213eeee87b8b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.320380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a503a-d3cb-4af3-9a71-a86f5d4d4101 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.327060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1a85d-545b-44ec-9157-8ad09d6d2da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.327999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a85d-545b-44ec-9157-8ad09d6d2da0 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.328287 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a85d-545b-44ec-9157-8ad09d6d2da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.328561 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a85d-545b-44ec-9157-8ad09d6d2da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.329323 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1a85d-545b-44ec-9157-8ad09d6d2da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.330919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1a85d-545b-44ec-9157-8ad09d6d2da0 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.336473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 806/1596] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:24 2022] POST /identity/v3/users => generated 294 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:24.336473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530f610-f3a4-463b-86b2-213eeee87b8b admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.342895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 791/1597] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:24.344943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530f610-f3a4-463b-86b2-213eeee87b8b admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.346716 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1a85d-545b-44ec-9157-8ad09d6d2da0 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.349625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98cf18-1418-48be-a4b4-8047299847dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.350540 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98cf18-1418-48be-a4b4-8047299847dc admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.350828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98cf18-1418-48be-a4b4-8047299847dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.351107 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98cf18-1418-48be-a4b4-8047299847dc admin admin] PATH_INFO: `/v3/projects/4fe52006c5034bbf8de92b8aae171d3c/users/bc6b0bfbda8944369d1a83b52b400f64/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.358088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1a85d-545b-44ec-9157-8ad09d6d2da0 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.364265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84adcd6b-4322-417d-ba51-9b87dd5b02b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.364951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adcd6b-4322-417d-ba51-9b87dd5b02b4 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.365232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adcd6b-4322-417d-ba51-9b87dd5b02b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.365514 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adcd6b-4322-417d-ba51-9b87dd5b02b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.366253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adcd6b-4322-417d-ba51-9b87dd5b02b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.367898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adcd6b-4322-417d-ba51-9b87dd5b02b4 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.369967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98cf18-1418-48be-a4b4-8047299847dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fe52006c5034bbf8de92b8aae171d3c, user_id=bc6b0bfbda8944369d1a83b52b400f64, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.372663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98cf18-1418-48be-a4b4-8047299847dc admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.380728 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 792/1598] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:24.381720 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84adcd6b-4322-417d-ba51-9b87dd5b02b4 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.387502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 807/1599] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:24 2022] PUT /identity/v3/projects/4fe52006c5034bbf8de92b8aae171d3c/users/bc6b0bfbda8944369d1a83b52b400f64/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:24.391154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1fafa-bc8a-4b60-bc0e-5d66370a3b85 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.391154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1fafa-bc8a-4b60-bc0e-5d66370a3b85 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.391154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1fafa-bc8a-4b60-bc0e-5d66370a3b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.393984 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84adcd6b-4322-417d-ba51-9b87dd5b02b4 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.420550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-14d1fafa-bc8a-4b60-bc0e-5d66370a3b85 None None] MFA Rules not processed for user `82764d7145a24ccc841052a776d1e4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:24.421767 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-14d1fafa-bc8a-4b60-bc0e-5d66370a3b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:24.440426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:24.440642 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:24.457021 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ee96c-578e-4b5f-91f1-c819f1aece7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.457855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ee96c-578e-4b5f-91f1-c819f1aece7a admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.458172 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ee96c-578e-4b5f-91f1-c819f1aece7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.458451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ee96c-578e-4b5f-91f1-c819f1aece7a admin admin] PATH_INFO: `/v3/projects/d9e9d8f8b55c4ef586252456ea195266/users/24e4efb87cd34008a1b83c0ea0cce5aa/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.474191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ee96c-578e-4b5f-91f1-c819f1aece7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e9d8f8b55c4ef586252456ea195266, user_id=24e4efb87cd34008a1b83c0ea0cce5aa, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.476941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ee96c-578e-4b5f-91f1-c819f1aece7a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.491938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 808/1600] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:24.492229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 793/1601] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:24 2022] PUT /identity/v3/projects/d9e9d8f8b55c4ef586252456ea195266/users/24e4efb87cd34008a1b83c0ea0cce5aa/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:24.493001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ee96c-578e-4b5f-91f1-c819f1aece7a admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.493114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1fafa-bc8a-4b60-bc0e-5d66370a3b85 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.505433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1fafa-bc8a-4b60-bc0e-5d66370a3b85 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.569236 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedb8c5-162b-4009-aed4-bb86a3596ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.569904 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedb8c5-162b-4009-aed4-bb86a3596ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.570190 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedb8c5-162b-4009-aed4-bb86a3596ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.570466 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedb8c5-162b-4009-aed4-bb86a3596ab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.570564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ee96c-578e-4b5f-91f1-c819f1aece7a admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.571162 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bedb8c5-162b-4009-aed4-bb86a3596ab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.573795 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bedb8c5-162b-4009-aed4-bb86a3596ab2 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.575774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebe5ec-c7d4-4766-b5ce-cd9e56a5ad40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.576438 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebe5ec-c7d4-4766-b5ce-cd9e56a5ad40 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.576718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebe5ec-c7d4-4766-b5ce-cd9e56a5ad40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.576992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebe5ec-c7d4-4766-b5ce-cd9e56a5ad40 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.577729 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebe5ec-c7d4-4766-b5ce-cd9e56a5ad40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.579305 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebe5ec-c7d4-4766-b5ce-cd9e56a5ad40 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.590049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 794/1602] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:24.591020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebe5ec-c7d4-4766-b5ce-cd9e56a5ad40 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.601326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebe5ec-c7d4-4766-b5ce-cd9e56a5ad40 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.604459 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 809/1603] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:24 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:24.605515 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedb8c5-162b-4009-aed4-bb86a3596ab2 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.607190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac7bf8-c125-4fa7-a7d2-8126b12983ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.608106 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac7bf8-c125-4fa7-a7d2-8126b12983ae admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.608394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac7bf8-c125-4fa7-a7d2-8126b12983ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.608673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac7bf8-c125-4fa7-a7d2-8126b12983ae admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.609435 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac7bf8-c125-4fa7-a7d2-8126b12983ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.611294 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac7bf8-c125-4fa7-a7d2-8126b12983ae admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.624797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 795/1604] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:24.624797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac7bf8-c125-4fa7-a7d2-8126b12983ae admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.632154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac7bf8-c125-4fa7-a7d2-8126b12983ae admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.669431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedb8c5-162b-4009-aed4-bb86a3596ab2 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.674113 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40a9a9-534d-4388-9e03-8af998a2a532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.674781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40a9a9-534d-4388-9e03-8af998a2a532 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.675063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40a9a9-534d-4388-9e03-8af998a2a532 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.675337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40a9a9-534d-4388-9e03-8af998a2a532 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.676104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40a9a9-534d-4388-9e03-8af998a2a532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.677699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40a9a9-534d-4388-9e03-8af998a2a532 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.688090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 810/1605] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:24.694248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e090efe-b2dc-41a9-8bce-3c2d93b2b7ca None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.694248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e090efe-b2dc-41a9-8bce-3c2d93b2b7ca None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.694248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e090efe-b2dc-41a9-8bce-3c2d93b2b7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.694563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889be36-bf1b-4c5d-9297-2c88042e3e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.694563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889be36-bf1b-4c5d-9297-2c88042e3e4f admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.694563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889be36-bf1b-4c5d-9297-2c88042e3e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.694563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889be36-bf1b-4c5d-9297-2c88042e3e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.694929 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889be36-bf1b-4c5d-9297-2c88042e3e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.694929 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889be36-bf1b-4c5d-9297-2c88042e3e4f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.706780 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 796/1606] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:24.708176 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db7f0c-f33f-46e6-8cb3-4991d5218440 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.708409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db7f0c-f33f-46e6-8cb3-4991d5218440 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.708508 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-2e090efe-b2dc-41a9-8bce-3c2d93b2b7ca None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:24.708688 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db7f0c-f33f-46e6-8cb3-4991d5218440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.709815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-2e090efe-b2dc-41a9-8bce-3c2d93b2b7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:24.717624 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 811/1607] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:24.719038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe5cb4-d792-4dcb-ad80-da807708bcbd None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.719325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe5cb4-d792-4dcb-ad80-da807708bcbd None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.719602 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe5cb4-d792-4dcb-ad80-da807708bcbd None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.721568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 812/1608] 10.222.0.60 () {54 vars in 962 bytes} [Thu Jul 21 20:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:52:24.722036 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe5cb4-d792-4dcb-ad80-da807708bcbd None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.732254 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c4db7f0c-f33f-46e6-8cb3-4991d5218440 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:24.734534 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c4db7f0c-f33f-46e6-8cb3-4991d5218440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:24.739940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe5cb4-d792-4dcb-ad80-da807708bcbd None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.746708 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c29918-3a8e-418f-8508-11b0830a35a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.750375 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 797/1609] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:24.750524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c29918-3a8e-418f-8508-11b0830a35a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.750524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c29918-3a8e-418f-8508-11b0830a35a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.750727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db7f0c-f33f-46e6-8cb3-4991d5218440 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.751310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c29918-3a8e-418f-8508-11b0830a35a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.751310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c29918-3a8e-418f-8508-11b0830a35a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.751310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c29918-3a8e-418f-8508-11b0830a35a6 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.761638 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db7f0c-f33f-46e6-8cb3-4991d5218440 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.764839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 813/1610] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:24.765556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c29918-3a8e-418f-8508-11b0830a35a6 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.766810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f8ba8-d64b-4b6a-9797-e7f5240a0df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.767509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f8ba8-d64b-4b6a-9797-e7f5240a0df4 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.767836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f8ba8-d64b-4b6a-9797-e7f5240a0df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.768142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f8ba8-d64b-4b6a-9797-e7f5240a0df4 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.768840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f8ba8-d64b-4b6a-9797-e7f5240a0df4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.770760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f8ba8-d64b-4b6a-9797-e7f5240a0df4 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.777236 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c29918-3a8e-418f-8508-11b0830a35a6 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.782053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110a0c6-b9c7-40ff-a7d0-563e318eba1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.782817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110a0c6-b9c7-40ff-a7d0-563e318eba1c admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.783139 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110a0c6-b9c7-40ff-a7d0-563e318eba1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.783469 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110a0c6-b9c7-40ff-a7d0-563e318eba1c admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.784297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110a0c6-b9c7-40ff-a7d0-563e318eba1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.786043 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110a0c6-b9c7-40ff-a7d0-563e318eba1c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.791080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 798/1611] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:24 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:24.792476 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f8ba8-d64b-4b6a-9797-e7f5240a0df4 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.800211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 814/1612] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:24.800983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110a0c6-b9c7-40ff-a7d0-563e318eba1c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.805458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f8ba8-d64b-4b6a-9797-e7f5240a0df4 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.811211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110a0c6-b9c7-40ff-a7d0-563e318eba1c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.811783 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfc0d7-2a1e-4e62-8fb0-15fdb5b1f598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.812678 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfc0d7-2a1e-4e62-8fb0-15fdb5b1f598 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.812959 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfc0d7-2a1e-4e62-8fb0-15fdb5b1f598 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.813233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfc0d7-2a1e-4e62-8fb0-15fdb5b1f598 admin admin] PATH_INFO: `/v3/projects/3dfdd7ea8c3a4c36a22b17049e7a1669/users/8a7bbf873df0414f8effa9f8efaf2b00/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.815858 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4af69e-bfbf-4fa4-aa4f-6425f02d9d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.816720 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4af69e-bfbf-4fa4-aa4f-6425f02d9d4e admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.817000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4af69e-bfbf-4fa4-aa4f-6425f02d9d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.817277 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4af69e-bfbf-4fa4-aa4f-6425f02d9d4e admin admin] PATH_INFO: `/v3/projects/1e9e2fb1afd34141bb9cb2d2de1694ab/users/abe04d8702cf47c788a9e309411c3273/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.825648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:24.829190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dfc0d7-2a1e-4e62-8fb0-15fdb5b1f598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfdd7ea8c3a4c36a22b17049e7a1669, user_id=8a7bbf873df0414f8effa9f8efaf2b00, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.832018 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dfc0d7-2a1e-4e62-8fb0-15fdb5b1f598 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.833989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4af69e-bfbf-4fa4-aa4f-6425f02d9d4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9e2fb1afd34141bb9cb2d2de1694ab, user_id=abe04d8702cf47c788a9e309411c3273, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.837625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4af69e-bfbf-4fa4-aa4f-6425f02d9d4e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.842886 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:24.848334 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 799/1613] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:24 2022] PUT /identity/v3/projects/3dfdd7ea8c3a4c36a22b17049e7a1669/users/8a7bbf873df0414f8effa9f8efaf2b00/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:24.849784 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448ef49-2f33-4416-8df6-faa93b3ab568 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.850253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448ef49-2f33-4416-8df6-faa93b3ab568 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.850897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448ef49-2f33-4416-8df6-faa93b3ab568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.860509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 815/1614] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:24 2022] PUT /identity/v3/projects/1e9e2fb1afd34141bb9cb2d2de1694ab/users/abe04d8702cf47c788a9e309411c3273/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:24.861612 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4af69e-bfbf-4fa4-aa4f-6425f02d9d4e admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.890932 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d448ef49-2f33-4416-8df6-faa93b3ab568 None None] MFA Rules not processed for user `77812b356c974d2b80102217956169db`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:24.892192 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d448ef49-2f33-4416-8df6-faa93b3ab568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:24.917872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4af69e-bfbf-4fa4-aa4f-6425f02d9d4e admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.921691 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fa5b8-3f6c-4d5f-8349-708428522b41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.922330 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fa5b8-3f6c-4d5f-8349-708428522b41 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.922620 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fa5b8-3f6c-4d5f-8349-708428522b41 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.922905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fa5b8-3f6c-4d5f-8349-708428522b41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.960835 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 800/1615] 10.222.0.60 () {58 vars in 1059 bytes} [Thu Jul 21 20:52:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:24.961601 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448ef49-2f33-4416-8df6-faa93b3ab568 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:24.971137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448ef49-2f33-4416-8df6-faa93b3ab568 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:24.974742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfbe62-594b-482d-9e08-a7a13ede994f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:24.975382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfbe62-594b-482d-9e08-a7a13ede994f service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:24.975951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfbe62-594b-482d-9e08-a7a13ede994f service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:24.975951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfbe62-594b-482d-9e08-a7a13ede994f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:24.987199 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fa5b8-3f6c-4d5f-8349-708428522b41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:24.988672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fa5b8-3f6c-4d5f-8349-708428522b41 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:24.998498 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 816/1616] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:24.999281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fa5b8-3f6c-4d5f-8349-708428522b41 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:25.009262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fa5b8-3f6c-4d5f-8349-708428522b41 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:25.015406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e207f4-927e-449c-bd71-26e3df1d3615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:25.016325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e207f4-927e-449c-bd71-26e3df1d3615 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:25.016609 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e207f4-927e-449c-bd71-26e3df1d3615 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:25.016885 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e207f4-927e-449c-bd71-26e3df1d3615 admin admin] PATH_INFO: `/v3/projects/cda29a2ffb81483286571baa4f16d2e4/users/ab080027ba304297bd7a63d546b2bd61/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:25.035029 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e207f4-927e-449c-bd71-26e3df1d3615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda29a2ffb81483286571baa4f16d2e4, user_id=ab080027ba304297bd7a63d546b2bd61, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:25.037811 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e207f4-927e-449c-bd71-26e3df1d3615 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:25.043593 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfbe62-594b-482d-9e08-a7a13ede994f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:25.045185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfbe62-594b-482d-9e08-a7a13ede994f service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:25.053367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 817/1617] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:24 2022] PUT /identity/v3/projects/cda29a2ffb81483286571baa4f16d2e4/users/ab080027ba304297bd7a63d546b2bd61/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:25.054292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e207f4-927e-449c-bd71-26e3df1d3615 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:25.075922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 801/1618] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:25.077949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfbe62-594b-482d-9e08-a7a13ede994f service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:25.095359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfbe62-594b-482d-9e08-a7a13ede994f service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:25.143693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e207f4-927e-449c-bd71-26e3df1d3615 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:25.152582 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442360de-d873-49b6-9421-48f784147bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:25.153366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442360de-d873-49b6-9421-48f784147bad admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:25.154142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442360de-d873-49b6-9421-48f784147bad admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:25.154142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442360de-d873-49b6-9421-48f784147bad admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:25.154931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442360de-d873-49b6-9421-48f784147bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:25.156895 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442360de-d873-49b6-9421-48f784147bad admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:25.183996 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 818/1619] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:25.185970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709789c-5de6-4645-8630-2e1789a3eae1 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:25.186764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709789c-5de6-4645-8630-2e1789a3eae1 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:25.187142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709789c-5de6-4645-8630-2e1789a3eae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:25.197946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af6b14-2205-4064-af3b-8b4d24133da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:25.199770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af6b14-2205-4064-af3b-8b4d24133da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:25.200262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af6b14-2205-4064-af3b-8b4d24133da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:25.200793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af6b14-2205-4064-af3b-8b4d24133da1 admin admin] PATH_INFO: `/v3/projects/c312164b3fdd40719fcdd94ed6bfd187/users/3a0d900f08d54098870ebbfa281e2a65/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:25.212469 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:25.227442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-4709789c-5de6-4645-8630-2e1789a3eae1 None None] MFA Rules not processed for user `8a7bbf873df0414f8effa9f8efaf2b00`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:25.227743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af6b14-2205-4064-af3b-8b4d24133da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c312164b3fdd40719fcdd94ed6bfd187, user_id=3a0d900f08d54098870ebbfa281e2a65, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:25.229499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-4709789c-5de6-4645-8630-2e1789a3eae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:25.231463 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af6b14-2205-4064-af3b-8b4d24133da1 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:25.236534 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:25.258054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 802/1620] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:25 2022] PUT /identity/v3/projects/c312164b3fdd40719fcdd94ed6bfd187/users/3a0d900f08d54098870ebbfa281e2a65/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:25.261204 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b123470-995f-4e72-b35a-f163dc4c97ba None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:25.261492 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b123470-995f-4e72-b35a-f163dc4c97ba None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:25.261760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b123470-995f-4e72-b35a-f163dc4c97ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:25.311855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-3b123470-995f-4e72-b35a-f163dc4c97ba None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:25.313672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-3b123470-995f-4e72-b35a-f163dc4c97ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:25.334570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 819/1621] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:25.335865 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709789c-5de6-4645-8630-2e1789a3eae1 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:25.412459 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 803/1622] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:25.414029 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908fa38-fdfe-434e-adcc-86ff7885b3c0 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:25.414389 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908fa38-fdfe-434e-adcc-86ff7885b3c0 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:25.414731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908fa38-fdfe-434e-adcc-86ff7885b3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:25.456638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709789c-5de6-4645-8630-2e1789a3eae1 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:25.462765 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-1908fa38-fdfe-434e-adcc-86ff7885b3c0 None None] MFA Rules not processed for user `ab080027ba304297bd7a63d546b2bd61`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:25.467926 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1908fa38-fdfe-434e-adcc-86ff7885b3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:25.572300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c449e6-8291-46da-aae4-72e47df7b814 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:25.573800 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c449e6-8291-46da-aae4-72e47df7b814 service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:25.574503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c449e6-8291-46da-aae4-72e47df7b814 service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:25.574938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c449e6-8291-46da-aae4-72e47df7b814 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:25.586912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 804/1623] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:25.589131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908fa38-fdfe-434e-adcc-86ff7885b3c0 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:25.604759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908fa38-fdfe-434e-adcc-86ff7885b3c0 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:25.611545 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6a696-0738-4def-80e9-59f543353ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:25.612625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6a696-0738-4def-80e9-59f543353ef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:25.613002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6a696-0738-4def-80e9-59f543353ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:25.613390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6a696-0738-4def-80e9-59f543353ef6 admin admin] PATH_INFO: `/v3/projects/8f7faeee165a4509a4c372e1c9de32cf/users/833cb739101e4888a7e29b30c2839caf/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:25.624314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:25.632743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6a696-0738-4def-80e9-59f543353ef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f7faeee165a4509a4c372e1c9de32cf, user_id=833cb739101e4888a7e29b30c2839caf, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:25.637598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6a696-0738-4def-80e9-59f543353ef6 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:25.656600 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 805/1624] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:25 2022] PUT /identity/v3/projects/8f7faeee165a4509a4c372e1c9de32cf/users/833cb739101e4888a7e29b30c2839caf/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:25.658313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1375ba2-f729-448a-b149-a1061e0b5805 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:25.658709 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1375ba2-f729-448a-b149-a1061e0b5805 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:25.659746 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1375ba2-f729-448a-b149-a1061e0b5805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:25.664699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c449e6-8291-46da-aae4-72e47df7b814 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:25.667619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c449e6-8291-46da-aae4-72e47df7b814 service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:25.683369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-a1375ba2-f729-448a-b149-a1061e0b5805 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:25.684972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a1375ba2-f729-448a-b149-a1061e0b5805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:25.686902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 820/1625] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:25.687740 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c449e6-8291-46da-aae4-72e47df7b814 service nova] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:25.703895 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c449e6-8291-46da-aae4-72e47df7b814 service nova] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:25.773715 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 806/1626] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:25.774446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1375ba2-f729-448a-b149-a1061e0b5805 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:25.779552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607cc8a-6aeb-4c1d-915e-47021402d8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:25.779814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:25.780463 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607cc8a-6aeb-4c1d-915e-47021402d8fa admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:25.780859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607cc8a-6aeb-4c1d-915e-47021402d8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:25.781427 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607cc8a-6aeb-4c1d-915e-47021402d8fa admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:25.782203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607cc8a-6aeb-4c1d-915e-47021402d8fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:25.784476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607cc8a-6aeb-4c1d-915e-47021402d8fa admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:25.831072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 821/1627] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:25 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:25.835766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ca729-f015-4fb6-a58a-3c628fe5d932 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:25.836067 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ca729-f015-4fb6-a58a-3c628fe5d932 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:25.836352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ca729-f015-4fb6-a58a-3c628fe5d932 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:25.837711 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 822/1628] 10.222.0.60 () {54 vars in 962 bytes} [Thu Jul 21 20:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:52:25.839282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ca729-f015-4fb6-a58a-3c628fe5d932 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:25.885873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1375ba2-f729-448a-b149-a1061e0b5805 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:25.924136 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ca729-f015-4fb6-a58a-3c628fe5d932 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:25.931835 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f166c7-cfb3-464f-89fd-c3b01ca0351b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:25.933103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f166c7-cfb3-464f-89fd-c3b01ca0351b admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:25.933991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f166c7-cfb3-464f-89fd-c3b01ca0351b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:25.934344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f166c7-cfb3-464f-89fd-c3b01ca0351b admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:25.935655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f166c7-cfb3-464f-89fd-c3b01ca0351b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:25.938437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f166c7-cfb3-464f-89fd-c3b01ca0351b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:25.956906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 823/1629] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:25.956906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f166c7-cfb3-464f-89fd-c3b01ca0351b admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:25.971633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabdf6a-46ae-456f-88e0-dbff3c98702a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:25.972326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabdf6a-46ae-456f-88e0-dbff3c98702a service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:25.972610 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabdf6a-46ae-456f-88e0-dbff3c98702a service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:25.972884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabdf6a-46ae-456f-88e0-dbff3c98702a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:25.983950 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabdf6a-46ae-456f-88e0-dbff3c98702a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:25.986764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabdf6a-46ae-456f-88e0-dbff3c98702a service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:26.001538 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 807/1630] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:26.002640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabdf6a-46ae-456f-88e0-dbff3c98702a service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:26.022665 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabdf6a-46ae-456f-88e0-dbff3c98702a service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:26.033994 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f2225-06d8-4714-842b-8894715206a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:26.036033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f2225-06d8-4714-842b-8894715206a8 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:26.037604 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f2225-06d8-4714-842b-8894715206a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:26.038253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f2225-06d8-4714-842b-8894715206a8 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:26.039506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f2225-06d8-4714-842b-8894715206a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:26.044017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f2225-06d8-4714-842b-8894715206a8 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:26.049041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f166c7-cfb3-464f-89fd-c3b01ca0351b admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:26.056525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eed262-01f4-4156-b66e-e7483bb71bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:26.057505 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eed262-01f4-4156-b66e-e7483bb71bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:26.057800 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eed262-01f4-4156-b66e-e7483bb71bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:26.058078 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eed262-01f4-4156-b66e-e7483bb71bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:26.075644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eed262-01f4-4156-b66e-e7483bb71bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:26.077292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eed262-01f4-4156-b66e-e7483bb71bf1 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:26.085359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:26.090349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 808/1631] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:26 2022] POST /identity/v3/users => generated 293 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:26.092471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fdc7b-4e4c-4c18-8844-6a7da5750fed None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:26.092893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fdc7b-4e4c-4c18-8844-6a7da5750fed None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:26.093582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fdc7b-4e4c-4c18-8844-6a7da5750fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:26.100588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 824/1632] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:26.101567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eed262-01f4-4156-b66e-e7483bb71bf1 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:26.113857 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eed262-01f4-4156-b66e-e7483bb71bf1 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:26.118990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-f54fdc7b-4e4c-4c18-8844-6a7da5750fed None None] MFA Rules not processed for user `77812b356c974d2b80102217956169db`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:26.119150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47159d-f48e-42fe-b8e5-d8a36d4e1904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:26.120077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47159d-f48e-42fe-b8e5-d8a36d4e1904 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:26.120389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47159d-f48e-42fe-b8e5-d8a36d4e1904 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:26.120801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47159d-f48e-42fe-b8e5-d8a36d4e1904 admin admin] PATH_INFO: `/v3/projects/8f7faeee165a4509a4c372e1c9de32cf/users/833cb739101e4888a7e29b30c2839caf/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:26.120928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f54fdc7b-4e4c-4c18-8844-6a7da5750fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:26.123145 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47159d-f48e-42fe-b8e5-d8a36d4e1904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f7faeee165a4509a4c372e1c9de32cf, user_id=833cb739101e4888a7e29b30c2839caf, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:26.126969 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47159d-f48e-42fe-b8e5-d8a36d4e1904 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:26.152290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 825/1633] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:26 2022] PUT /identity/v3/projects/8f7faeee165a4509a4c372e1c9de32cf/users/833cb739101e4888a7e29b30c2839caf/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:26.152290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221b3af-b68f-490f-b6eb-c81540ca6b35 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:26.152290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221b3af-b68f-490f-b6eb-c81540ca6b35 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:26.152290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221b3af-b68f-490f-b6eb-c81540ca6b35 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:26.152290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 826/1634] 10.222.0.60 () {54 vars in 912 bytes} [Thu Jul 21 20:52:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:52:26.152290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221b3af-b68f-490f-b6eb-c81540ca6b35 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:26.239995 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 809/1635] 10.222.0.60 () {58 vars in 1059 bytes} [Thu Jul 21 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:26.257432 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba151e-1eef-4b58-9bbe-e3d0fbc843e9 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:26.257978 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba151e-1eef-4b58-9bbe-e3d0fbc843e9 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:26.258451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba151e-1eef-4b58-9bbe-e3d0fbc843e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:26.284310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-4dba151e-1eef-4b58-9bbe-e3d0fbc843e9 None None] MFA Rules not processed for user `77812b356c974d2b80102217956169db`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:26.286466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-4dba151e-1eef-4b58-9bbe-e3d0fbc843e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:26.302828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221b3af-b68f-490f-b6eb-c81540ca6b35 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:26.312819 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3bedb-ee72-46c7-bf5a-18b669114f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:26.313552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3bedb-ee72-46c7-bf5a-18b669114f7f admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:26.313781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3bedb-ee72-46c7-bf5a-18b669114f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:26.314060 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3bedb-ee72-46c7-bf5a-18b669114f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:26.314804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3bedb-ee72-46c7-bf5a-18b669114f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:26.316457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3bedb-ee72-46c7-bf5a-18b669114f7f admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:26.339232 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 827/1636] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:26.340735 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:26.342779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753f488-9a3f-4fa6-af2a-1552b05029f5 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:26.343687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753f488-9a3f-4fa6-af2a-1552b05029f5 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:26.344343 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753f488-9a3f-4fa6-af2a-1552b05029f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:26.392289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-2753f488-9a3f-4fa6-af2a-1552b05029f5 None None] MFA Rules not processed for user `833cb739101e4888a7e29b30c2839caf`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:26.394792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-2753f488-9a3f-4fa6-af2a-1552b05029f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:26.404921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 810/1637] 10.222.0.60 () {58 vars in 1009 bytes} [Thu Jul 21 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:26.408318 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba151e-1eef-4b58-9bbe-e3d0fbc843e9 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:26.436317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba151e-1eef-4b58-9bbe-e3d0fbc843e9 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:26.444669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05656801-ccfe-4b2e-8c89-78346e76c9c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:26.446207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05656801-ccfe-4b2e-8c89-78346e76c9c0 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:26.446844 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05656801-ccfe-4b2e-8c89-78346e76c9c0 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:26.447381 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05656801-ccfe-4b2e-8c89-78346e76c9c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:26.536547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 828/1638] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:26.536547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753f488-9a3f-4fa6-af2a-1552b05029f5 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:26.547094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05656801-ccfe-4b2e-8c89-78346e76c9c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:26.547908 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753f488-9a3f-4fa6-af2a-1552b05029f5 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:26.548860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05656801-ccfe-4b2e-8c89-78346e76c9c0 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:26.553470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cb1f7-1057-4339-9e76-57b1bc6482d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:26.554557 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cb1f7-1057-4339-9e76-57b1bc6482d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:26.554843 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cb1f7-1057-4339-9e76-57b1bc6482d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:26.555119 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cb1f7-1057-4339-9e76-57b1bc6482d7 admin admin] PATH_INFO: `/v3/projects/b221cf0ea3554c68946ad0014f2cd40a/users/44d9d1fce96746f6bebf01335d24d09f/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:26.578187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778cb1f7-1057-4339-9e76-57b1bc6482d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b221cf0ea3554c68946ad0014f2cd40a, user_id=44d9d1fce96746f6bebf01335d24d09f, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:26.583626 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778cb1f7-1057-4339-9e76-57b1bc6482d7 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:26.584972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 811/1639] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:26.589244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05656801-ccfe-4b2e-8c89-78346e76c9c0 service nova] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:26.614588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 829/1640] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:26 2022] PUT /identity/v3/projects/b221cf0ea3554c68946ad0014f2cd40a/users/44d9d1fce96746f6bebf01335d24d09f/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:26.616128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cb1f7-1057-4339-9e76-57b1bc6482d7 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:26.748504 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:26.761123 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05656801-ccfe-4b2e-8c89-78346e76c9c0 service nova] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:26.774686 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730f168-8cbf-4d18-aee6-e4506dc81c3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:26.775375 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730f168-8cbf-4d18-aee6-e4506dc81c3a service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:26.776253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730f168-8cbf-4d18-aee6-e4506dc81c3a service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:26.776537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730f168-8cbf-4d18-aee6-e4506dc81c3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:26.777967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cb1f7-1057-4339-9e76-57b1bc6482d7 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:26.786457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8870c1-df11-4be8-8d46-cee5671c8c9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:26.788316 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8870c1-df11-4be8-8d46-cee5671c8c9f service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:26.788581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8870c1-df11-4be8-8d46-cee5671c8c9f service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:26.788864 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8870c1-df11-4be8-8d46-cee5671c8c9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:26.941536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8870c1-df11-4be8-8d46-cee5671c8c9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:26.943519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8870c1-df11-4be8-8d46-cee5671c8c9f service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:26.959836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730f168-8cbf-4d18-aee6-e4506dc81c3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:26.961391 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730f168-8cbf-4d18-aee6-e4506dc81c3a service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:26.966922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 830/1641] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:26.967863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8870c1-df11-4be8-8d46-cee5671c8c9f service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:26.983580 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 812/1642] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:27.116224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8870c1-df11-4be8-8d46-cee5671c8c9f service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:27.128459 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b706f-993a-4bba-9e93-442cdb105b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:27.129156 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b706f-993a-4bba-9e93-442cdb105b2a admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:27.129498 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b706f-993a-4bba-9e93-442cdb105b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:27.129772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b706f-993a-4bba-9e93-442cdb105b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:27.130667 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b706f-993a-4bba-9e93-442cdb105b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:27.134949 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b706f-993a-4bba-9e93-442cdb105b2a admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:27.154436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730f168-8cbf-4d18-aee6-e4506dc81c3a service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:27.157162 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 831/1643] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:27.171438 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730f168-8cbf-4d18-aee6-e4506dc81c3a service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:27.185788 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c5688-7493-4f8c-9dfd-1eec49520f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:27.186703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c5688-7493-4f8c-9dfd-1eec49520f49 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:27.186984 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c5688-7493-4f8c-9dfd-1eec49520f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:27.187258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c5688-7493-4f8c-9dfd-1eec49520f49 admin admin] PATH_INFO: `/v3/projects/b221cf0ea3554c68946ad0014f2cd40a/users/44d9d1fce96746f6bebf01335d24d09f/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:27.198850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c5688-7493-4f8c-9dfd-1eec49520f49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b221cf0ea3554c68946ad0014f2cd40a, user_id=44d9d1fce96746f6bebf01335d24d09f, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:27.209441 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c5688-7493-4f8c-9dfd-1eec49520f49 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:27.210393 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:27.232354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 813/1644] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:27 2022] PUT /identity/v3/projects/b221cf0ea3554c68946ad0014f2cd40a/users/44d9d1fce96746f6bebf01335d24d09f/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:52:27.244960 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aa35d-8438-4377-9483-7ac0ed03c67b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:27.245665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aa35d-8438-4377-9483-7ac0ed03c67b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:27.246317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aa35d-8438-4377-9483-7ac0ed03c67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:27.315329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-b03aa35d-8438-4377-9483-7ac0ed03c67b None None] MFA Rules not processed for user `44d9d1fce96746f6bebf01335d24d09f`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:27.316786 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b03aa35d-8438-4377-9483-7ac0ed03c67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:27.357356 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16baefaf-7c44-4003-8c5b-0443644ac008 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:27.357934 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16baefaf-7c44-4003-8c5b-0443644ac008 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:27.358298 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16baefaf-7c44-4003-8c5b-0443644ac008 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:27.359843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 814/1645] 10.222.0.60 () {54 vars in 912 bytes} [Thu Jul 21 20:52:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:52:27.367529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca43606-4569-49cd-bbf4-c9a295832d3d None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:27.368335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca43606-4569-49cd-bbf4-c9a295832d3d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:27.368545 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca43606-4569-49cd-bbf4-c9a295832d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:27.429411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-5ca43606-4569-49cd-bbf4-c9a295832d3d None None] MFA Rules not processed for user `77812b356c974d2b80102217956169db`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:27.435760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5ca43606-4569-49cd-bbf4-c9a295832d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:27.529730 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 832/1646] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:27.605122 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03aa35d-8438-4377-9483-7ac0ed03c67b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:27.638486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 815/1647] 10.222.0.60 () {58 vars in 1009 bytes} [Thu Jul 21 20:52:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:27.639964 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:27.755198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03aa35d-8438-4377-9483-7ac0ed03c67b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:27.761080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd9cd2-710c-4f45-b82f-3a0f2e94771e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:27.762306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd9cd2-710c-4f45-b82f-3a0f2e94771e service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:27.762306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd9cd2-710c-4f45-b82f-3a0f2e94771e service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:27.762588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd9cd2-710c-4f45-b82f-3a0f2e94771e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:27.789846 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd9cd2-710c-4f45-b82f-3a0f2e94771e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:27.791922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd9cd2-710c-4f45-b82f-3a0f2e94771e service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:27.821452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 833/1648] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:27.937627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca43606-4569-49cd-bbf4-c9a295832d3d None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:27.955225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca43606-4569-49cd-bbf4-c9a295832d3d None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:27.959944 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0a1b2-38d4-48db-a258-bbb1e9ef84c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:27.960964 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0a1b2-38d4-48db-a258-bbb1e9ef84c1 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:27.961448 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0a1b2-38d4-48db-a258-bbb1e9ef84c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:27.961917 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0a1b2-38d4-48db-a258-bbb1e9ef84c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:27.977209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0a1b2-38d4-48db-a258-bbb1e9ef84c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:27.981642 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0a1b2-38d4-48db-a258-bbb1e9ef84c1 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:28.005352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 816/1649] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:28.677629 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd9cd2-710c-4f45-b82f-3a0f2e94771e service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:28.730310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0a1b2-38d4-48db-a258-bbb1e9ef84c1 service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:28.796209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd9cd2-710c-4f45-b82f-3a0f2e94771e service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:28.805205 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475fb7f-92f7-4dae-a653-51e4831fdcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:28.806387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475fb7f-92f7-4dae-a653-51e4831fdcb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:28.808209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475fb7f-92f7-4dae-a653-51e4831fdcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:28.810713 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475fb7f-92f7-4dae-a653-51e4831fdcb3 admin admin] PATH_INFO: `/v3/users/ab080027ba304297bd7a63d546b2bd61` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:28.814721 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475fb7f-92f7-4dae-a653-51e4831fdcb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab080027ba304297bd7a63d546b2bd61)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:28.820727 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475fb7f-92f7-4dae-a653-51e4831fdcb3 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:28.908185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0a1b2-38d4-48db-a258-bbb1e9ef84c1 service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:28.930217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d838f-3bf8-4c03-a443-d0b8ea5197d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:28.932651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d838f-3bf8-4c03-a443-d0b8ea5197d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:28.932947 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d838f-3bf8-4c03-a443-d0b8ea5197d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:28.933223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d838f-3bf8-4c03-a443-d0b8ea5197d7 admin admin] PATH_INFO: `/v3/users/06ab1297aa0d4fd1aa345280e4be7e1c` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:28.937909 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d838f-3bf8-4c03-a443-d0b8ea5197d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06ab1297aa0d4fd1aa345280e4be7e1c)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:28.940724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: WARNING py.warnings [None req-443d838f-3bf8-4c03-a443-d0b8ea5197d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:52:28.940724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: warnings.warn(msg) Jul 21 20:52:28.940724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: Jul 21 20:52:28.946648 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d838f-3bf8-4c03-a443-d0b8ea5197d7 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:28.960880 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e475fb7f-92f7-4dae-a653-51e4831fdcb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab080027ba304297bd7a63d546b2bd61'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:28.961711 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e475fb7f-92f7-4dae-a653-51e4831fdcb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab080027ba304297bd7a63d546b2bd61'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:29.008778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e475fb7f-92f7-4dae-a653-51e4831fdcb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab080027ba304297bd7a63d546b2bd61'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:29.025485 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e475fb7f-92f7-4dae-a653-51e4831fdcb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab080027ba304297bd7a63d546b2bd61'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:29.040118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e475fb7f-92f7-4dae-a653-51e4831fdcb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab080027ba304297bd7a63d546b2bd61'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:29.058993 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 834/1650] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:52:28 2022] DELETE /identity/v3/users/ab080027ba304297bd7a63d546b2bd61 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:29.064223 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a133f63-94eb-436e-854b-e70a6dd361be None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:29.065555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a133f63-94eb-436e-854b-e70a6dd361be None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:29.066028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a133f63-94eb-436e-854b-e70a6dd361be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:29.122386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-1a133f63-94eb-436e-854b-e70a6dd361be None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:29.123826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1a133f63-94eb-436e-854b-e70a6dd361be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:29.128916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-443d838f-3bf8-4c03-a443-d0b8ea5197d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06ab1297aa0d4fd1aa345280e4be7e1c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:29.147973 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-443d838f-3bf8-4c03-a443-d0b8ea5197d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06ab1297aa0d4fd1aa345280e4be7e1c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:29.149499 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-443d838f-3bf8-4c03-a443-d0b8ea5197d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06ab1297aa0d4fd1aa345280e4be7e1c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:29.179062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-443d838f-3bf8-4c03-a443-d0b8ea5197d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06ab1297aa0d4fd1aa345280e4be7e1c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:29.184375 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-443d838f-3bf8-4c03-a443-d0b8ea5197d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06ab1297aa0d4fd1aa345280e4be7e1c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:29.211978 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:29.220320 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:29.250685 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 817/1651] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:52:28 2022] DELETE /identity/v3/users/06ab1297aa0d4fd1aa345280e4be7e1c => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:29.252820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d838f-3bf8-4c03-a443-d0b8ea5197d7 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:29.256916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-443d838f-3bf8-4c03-a443-d0b8ea5197d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:29.306080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 835/1652] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:29.311171 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2ea57-5841-4b30-891d-38f02e34806b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:29.311639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2ea57-5841-4b30-891d-38f02e34806b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:29.312114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2ea57-5841-4b30-891d-38f02e34806b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:29.336412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-efa2ea57-5841-4b30-891d-38f02e34806b None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:29.337782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-efa2ea57-5841-4b30-891d-38f02e34806b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:29.446833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 836/1653] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:29.448262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2ea57-5841-4b30-891d-38f02e34806b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:29.449771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-efa2ea57-5841-4b30-891d-38f02e34806b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:29.455677 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d838f-3bf8-4c03-a443-d0b8ea5197d7 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:29.467249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d615dc-6153-4ae5-859b-40eeea8e909f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:29.468221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d615dc-6153-4ae5-859b-40eeea8e909f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:29.468599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d615dc-6153-4ae5-859b-40eeea8e909f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:29.471331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d615dc-6153-4ae5-859b-40eeea8e909f admin admin] PATH_INFO: `/v3/users/8a7bbf873df0414f8effa9f8efaf2b00` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:29.475462 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d615dc-6153-4ae5-859b-40eeea8e909f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a7bbf873df0414f8effa9f8efaf2b00)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:29.477613 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d615dc-6153-4ae5-859b-40eeea8e909f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:29.566567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2ea57-5841-4b30-891d-38f02e34806b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:29.576945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787205a-764c-4b5b-98f5-93201f26b782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:29.577870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787205a-764c-4b5b-98f5-93201f26b782 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:29.578288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787205a-764c-4b5b-98f5-93201f26b782 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:29.578700 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787205a-764c-4b5b-98f5-93201f26b782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:29.618795 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787205a-764c-4b5b-98f5-93201f26b782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:29.625727 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787205a-764c-4b5b-98f5-93201f26b782 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:29.632334 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-40d615dc-6153-4ae5-859b-40eeea8e909f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a7bbf873df0414f8effa9f8efaf2b00'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:29.658186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 837/1654] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:29.659822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-40d615dc-6153-4ae5-859b-40eeea8e909f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a7bbf873df0414f8effa9f8efaf2b00'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:29.661002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-40d615dc-6153-4ae5-859b-40eeea8e909f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a7bbf873df0414f8effa9f8efaf2b00'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:29.669314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787205a-764c-4b5b-98f5-93201f26b782 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:29.688935 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-40d615dc-6153-4ae5-859b-40eeea8e909f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a7bbf873df0414f8effa9f8efaf2b00'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:29.691318 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-40d615dc-6153-4ae5-859b-40eeea8e909f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a7bbf873df0414f8effa9f8efaf2b00'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:29.695118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787205a-764c-4b5b-98f5-93201f26b782 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:29.711227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a42c3a-dda6-4b46-8344-95c606e57b28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:29.716673 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a42c3a-dda6-4b46-8344-95c606e57b28 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:29.716673 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a42c3a-dda6-4b46-8344-95c606e57b28 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:29.716673 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a42c3a-dda6-4b46-8344-95c606e57b28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:29.727009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e3a42c3a-dda6-4b46-8344-95c606e57b28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:29.767994 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 818/1655] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:52:29 2022] DELETE /identity/v3/users/8a7bbf873df0414f8effa9f8efaf2b00 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:29.777258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e0533-8a8b-4601-9a24-6d1530bd9b4a None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:29.777258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e0533-8a8b-4601-9a24-6d1530bd9b4a None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:29.777258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e0533-8a8b-4601-9a24-6d1530bd9b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:29.785298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a42c3a-dda6-4b46-8344-95c606e57b28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:29.789678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a42c3a-dda6-4b46-8344-95c606e57b28 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:29.820529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 838/1656] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:29.824915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cadfa-10ca-47ce-be4e-ea876ae5d538 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:29.840150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cadfa-10ca-47ce-be4e-ea876ae5d538 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:29.840699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cadfa-10ca-47ce-be4e-ea876ae5d538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:29.851999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-eb8e0533-8a8b-4601-9a24-6d1530bd9b4a None None] MFA Rules not processed for user `abe04d8702cf47c788a9e309411c3273`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:29.854703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-eb8e0533-8a8b-4601-9a24-6d1530bd9b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:29.887033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-8c9cadfa-10ca-47ce-be4e-ea876ae5d538 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:29.888278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8c9cadfa-10ca-47ce-be4e-ea876ae5d538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:29.976910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:29.987554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 819/1657] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:29 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:29.993696 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e0533-8a8b-4601-9a24-6d1530bd9b4a None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:29.997003 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-eb8e0533-8a8b-4601-9a24-6d1530bd9b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:30.051847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 839/1658] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:30.096848 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cadfa-10ca-47ce-be4e-ea876ae5d538 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:30.098822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8c9cadfa-10ca-47ce-be4e-ea876ae5d538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:30.103172 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e0533-8a8b-4601-9a24-6d1530bd9b4a None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:30.115164 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:30.119336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914fa72-d4fa-4073-bdc2-af974edd1954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:30.123347 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914fa72-d4fa-4073-bdc2-af974edd1954 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:30.123905 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914fa72-d4fa-4073-bdc2-af974edd1954 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:30.124268 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914fa72-d4fa-4073-bdc2-af974edd1954 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:30.125314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914fa72-d4fa-4073-bdc2-af974edd1954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:30.131798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914fa72-d4fa-4073-bdc2-af974edd1954 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:30.224844 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 820/1659] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:29 2022] POST /identity/v3/projects => generated 405 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:30.228683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914fa72-d4fa-4073-bdc2-af974edd1954 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:30.263163 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cadfa-10ca-47ce-be4e-ea876ae5d538 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:30.362562 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914fa72-d4fa-4073-bdc2-af974edd1954 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:30.371837 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba01ce-f15c-4b0a-8695-a2e77e8dc290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:30.372543 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba01ce-f15c-4b0a-8695-a2e77e8dc290 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:30.372832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba01ce-f15c-4b0a-8695-a2e77e8dc290 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:30.373110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba01ce-f15c-4b0a-8695-a2e77e8dc290 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:30.373887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba01ce-f15c-4b0a-8695-a2e77e8dc290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:30.377247 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba01ce-f15c-4b0a-8695-a2e77e8dc290 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:30.382467 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd0b55-153e-4d34-8146-a676ff957f8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:30.383143 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd0b55-153e-4d34-8146-a676ff957f8b service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:30.383428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd0b55-153e-4d34-8146-a676ff957f8b service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:30.383758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd0b55-153e-4d34-8146-a676ff957f8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:30.399102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd0b55-153e-4d34-8146-a676ff957f8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:30.408460 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd0b55-153e-4d34-8146-a676ff957f8b service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:30.436220 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 821/1660] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:30 2022] POST /identity/v3/users => generated 309 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:52:30.437675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba01ce-f15c-4b0a-8695-a2e77e8dc290 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:30.438511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 840/1661] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:30.457647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba01ce-f15c-4b0a-8695-a2e77e8dc290 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:30.462969 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0633072-3a16-41c3-b735-306e384b6405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:30.465819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0633072-3a16-41c3-b735-306e384b6405 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:30.466122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0633072-3a16-41c3-b735-306e384b6405 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:30.466408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0633072-3a16-41c3-b735-306e384b6405 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:30.468554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0633072-3a16-41c3-b735-306e384b6405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:30.470725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0633072-3a16-41c3-b735-306e384b6405 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:30.489906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 822/1662] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:52:30.495401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd0b55-153e-4d34-8146-a676ff957f8b service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:30.512340 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd0b55-153e-4d34-8146-a676ff957f8b service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:30.529833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a1c7c-f094-4f0a-beb8-a01f9579528f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:30.530779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a1c7c-f094-4f0a-beb8-a01f9579528f admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:30.531067 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a1c7c-f094-4f0a-beb8-a01f9579528f admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:30.531350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a1c7c-f094-4f0a-beb8-a01f9579528f admin admin] PATH_INFO: `/v3/projects/09f927548fe047218578d81e7cc92f8b/users/aba3ccf461484d9c84d3892eba8cef11/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:30.579426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a1c7c-f094-4f0a-beb8-a01f9579528f admin admin] RBAC: Authorizing `identity:create_grant(project_id=09f927548fe047218578d81e7cc92f8b, user_id=aba3ccf461484d9c84d3892eba8cef11, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:30.582995 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a1c7c-f094-4f0a-beb8-a01f9579528f admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:30.605215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 841/1663] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:30 2022] PUT /identity/v3/projects/09f927548fe047218578d81e7cc92f8b/users/aba3ccf461484d9c84d3892eba8cef11/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:30.608879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0633072-3a16-41c3-b735-306e384b6405 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:30.631072 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0633072-3a16-41c3-b735-306e384b6405 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:30.839925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8802783-d8a1-45e1-8f86-cdefd19006a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:30.840790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8802783-d8a1-45e1-8f86-cdefd19006a5 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:30.841081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8802783-d8a1-45e1-8f86-cdefd19006a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:30.841452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8802783-d8a1-45e1-8f86-cdefd19006a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:30.842210 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8802783-d8a1-45e1-8f86-cdefd19006a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:30.848507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8802783-d8a1-45e1-8f86-cdefd19006a5 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:30.861823 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:30.876233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a1c7c-f094-4f0a-beb8-a01f9579528f admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:30.888117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 823/1664] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:30.902532 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a1c7c-f094-4f0a-beb8-a01f9579528f admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:30.937722 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc69ea-c4d2-4e2b-a148-dadaf5cfe80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:30.941015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc69ea-c4d2-4e2b-a148-dadaf5cfe80c admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:30.942176 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc69ea-c4d2-4e2b-a148-dadaf5cfe80c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:30.944443 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc69ea-c4d2-4e2b-a148-dadaf5cfe80c admin admin] PATH_INFO: `/v3/projects/09f927548fe047218578d81e7cc92f8b/users/aba3ccf461484d9c84d3892eba8cef11/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:30.954197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc69ea-c4d2-4e2b-a148-dadaf5cfe80c admin admin] RBAC: Authorizing `identity:create_grant(project_id=09f927548fe047218578d81e7cc92f8b, user_id=aba3ccf461484d9c84d3892eba8cef11, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:30.964204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc69ea-c4d2-4e2b-a148-dadaf5cfe80c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:30.998021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 842/1665] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:30 2022] PUT /identity/v3/projects/09f927548fe047218578d81e7cc92f8b/users/aba3ccf461484d9c84d3892eba8cef11/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:31.684758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8802783-d8a1-45e1-8f86-cdefd19006a5 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:31.688171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e8802783-d8a1-45e1-8f86-cdefd19006a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:31.712461 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc69ea-c4d2-4e2b-a148-dadaf5cfe80c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:31.715799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-23cc69ea-c4d2-4e2b-a148-dadaf5cfe80c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:31.730330 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8802783-d8a1-45e1-8f86-cdefd19006a5 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:31.882937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c33ca-1465-49f6-afb2-594cffac6228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:31.884102 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c33ca-1465-49f6-afb2-594cffac6228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:31.885244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c33ca-1465-49f6-afb2-594cffac6228 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:31.885244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c33ca-1465-49f6-afb2-594cffac6228 admin admin] PATH_INFO: `/v3/projects/cda29a2ffb81483286571baa4f16d2e4` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:31.890279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c33ca-1465-49f6-afb2-594cffac6228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cda29a2ffb81483286571baa4f16d2e4)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:31.891984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc69ea-c4d2-4e2b-a148-dadaf5cfe80c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:31.893430 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c33ca-1465-49f6-afb2-594cffac6228 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:31.904570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:31.905704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:31.906134 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:31.906592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] PATH_INFO: `/v3/projects/511c4e2782a54c32b4f856a9cda6710a` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:31.907897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=511c4e2782a54c32b4f856a9cda6710a)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:31.908931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: WARNING py.warnings [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 20:52:31.908931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: warnings.warn(msg) Jul 21 20:52:31.908931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: Jul 21 20:52:31.910146 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:31.978623 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2a6c33ca-1465-49f6-afb2-594cffac6228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda29a2ffb81483286571baa4f16d2e4', 'request_id': 'req-2a6c33ca-1465-49f6-afb2-594cffac6228'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:31.987935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:31.995626 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2a6c33ca-1465-49f6-afb2-594cffac6228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda29a2ffb81483286571baa4f16d2e4', 'request_id': 'req-2a6c33ca-1465-49f6-afb2-594cffac6228'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.012467 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2a6c33ca-1465-49f6-afb2-594cffac6228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda29a2ffb81483286571baa4f16d2e4', 'request_id': 'req-2a6c33ca-1465-49f6-afb2-594cffac6228'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.032664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '511c4e2782a54c32b4f856a9cda6710a', 'request_id': 'req-85347082-40a7-49d6-9e1f-0c598938dc25'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.052575 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2a6c33ca-1465-49f6-afb2-594cffac6228 admin admin] The token cache is being invalidate because project cda29a2ffb81483286571baa4f16d2e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:52:32.052876 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2a6c33ca-1465-49f6-afb2-594cffac6228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.053727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2a6c33ca-1465-49f6-afb2-594cffac6228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.056255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 824/1666] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:52:31 2022] DELETE /identity/v3/projects/cda29a2ffb81483286571baa4f16d2e4 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:32.062437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e02ce-5b08-43eb-8112-371025c032cb None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:32.062437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e02ce-5b08-43eb-8112-371025c032cb None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:32.062437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e02ce-5b08-43eb-8112-371025c032cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:32.063085 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '511c4e2782a54c32b4f856a9cda6710a', 'request_id': 'req-85347082-40a7-49d6-9e1f-0c598938dc25'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.094987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '511c4e2782a54c32b4f856a9cda6710a', 'request_id': 'req-85347082-40a7-49d6-9e1f-0c598938dc25'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.123465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-910e02ce-5b08-43eb-8112-371025c032cb None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:32.125931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-910e02ce-5b08-43eb-8112-371025c032cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:32.126053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] The token cache is being invalidate because project 511c4e2782a54c32b4f856a9cda6710a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:52:32.126306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.128212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.130266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 843/1667] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:52:31 2022] DELETE /identity/v3/projects/511c4e2782a54c32b4f856a9cda6710a => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:32.134244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:32.136728 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:32.340261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 825/1668] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:32.344107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17379307-928c-4564-b284-eea4ca9c29eb None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:32.344403 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17379307-928c-4564-b284-eea4ca9c29eb None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:32.344686 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17379307-928c-4564-b284-eea4ca9c29eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:32.367450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85347082-40a7-49d6-9e1f-0c598938dc25 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:32.390923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d99f7b-894e-4f1c-bba8-4d5eaf9910c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:32.393355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d99f7b-894e-4f1c-bba8-4d5eaf9910c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:32.393645 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d99f7b-894e-4f1c-bba8-4d5eaf9910c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:32.393934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d99f7b-894e-4f1c-bba8-4d5eaf9910c4 admin admin] PATH_INFO: `/v3/users/833cb739101e4888a7e29b30c2839caf` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:32.397547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d99f7b-894e-4f1c-bba8-4d5eaf9910c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=833cb739101e4888a7e29b30c2839caf)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:32.402245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d99f7b-894e-4f1c-bba8-4d5eaf9910c4 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:32.418883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-17379307-928c-4564-b284-eea4ca9c29eb None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:32.418883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-17379307-928c-4564-b284-eea4ca9c29eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:32.450264 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 826/1669] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:32.451206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17379307-928c-4564-b284-eea4ca9c29eb None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:32.460868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-17379307-928c-4564-b284-eea4ca9c29eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:32.493666 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17379307-928c-4564-b284-eea4ca9c29eb None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:32.510771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fa28b-2935-42e4-9f9f-8d4a8c8033dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:32.511748 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fa28b-2935-42e4-9f9f-8d4a8c8033dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:32.512108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fa28b-2935-42e4-9f9f-8d4a8c8033dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:32.512391 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fa28b-2935-42e4-9f9f-8d4a8c8033dd admin admin] PATH_INFO: `/v3/users/82764d7145a24ccc841052a776d1e4a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:32.513725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fa28b-2935-42e4-9f9f-8d4a8c8033dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=82764d7145a24ccc841052a776d1e4a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:32.516006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fa28b-2935-42e4-9f9f-8d4a8c8033dd admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:32.520614 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e3d99f7b-894e-4f1c-bba8-4d5eaf9910c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '833cb739101e4888a7e29b30c2839caf'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.521585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e3d99f7b-894e-4f1c-bba8-4d5eaf9910c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '833cb739101e4888a7e29b30c2839caf'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.542558 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e3d99f7b-894e-4f1c-bba8-4d5eaf9910c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '833cb739101e4888a7e29b30c2839caf'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.554299 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e3d99f7b-894e-4f1c-bba8-4d5eaf9910c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '833cb739101e4888a7e29b30c2839caf'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.563397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:32.573343 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e3d99f7b-894e-4f1c-bba8-4d5eaf9910c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '833cb739101e4888a7e29b30c2839caf'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.600133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 844/1670] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:52:32 2022] DELETE /identity/v3/users/833cb739101e4888a7e29b30c2839caf => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:32.600133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d99f7b-894e-4f1c-bba8-4d5eaf9910c4 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:32.606172 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e3d99f7b-894e-4f1c-bba8-4d5eaf9910c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:32.611819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-1b2fa28b-2935-42e4-9f9f-8d4a8c8033dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82764d7145a24ccc841052a776d1e4a1'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.626225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-1b2fa28b-2935-42e4-9f9f-8d4a8c8033dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82764d7145a24ccc841052a776d1e4a1'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.627216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-1b2fa28b-2935-42e4-9f9f-8d4a8c8033dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82764d7145a24ccc841052a776d1e4a1'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.642323 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-1b2fa28b-2935-42e4-9f9f-8d4a8c8033dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82764d7145a24ccc841052a776d1e4a1'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.646608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-1b2fa28b-2935-42e4-9f9f-8d4a8c8033dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82764d7145a24ccc841052a776d1e4a1'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.680046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 827/1671] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:52:32 2022] DELETE /identity/v3/users/82764d7145a24ccc841052a776d1e4a1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:32.681312 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fa28b-2935-42e4-9f9f-8d4a8c8033dd admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:32.684096 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1b2fa28b-2935-42e4-9f9f-8d4a8c8033dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:32.695640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:32.750136 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d99f7b-894e-4f1c-bba8-4d5eaf9910c4 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:32.767893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fa28b-2935-42e4-9f9f-8d4a8c8033dd admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:32.773348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb6e98-3b64-4d80-9ac5-7758d305d6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:32.774257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb6e98-3b64-4d80-9ac5-7758d305d6ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:32.774546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb6e98-3b64-4d80-9ac5-7758d305d6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:32.774827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb6e98-3b64-4d80-9ac5-7758d305d6ca admin admin] PATH_INFO: `/v3/projects/3dfdd7ea8c3a4c36a22b17049e7a1669` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:32.776968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb6e98-3b64-4d80-9ac5-7758d305d6ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dfdd7ea8c3a4c36a22b17049e7a1669)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:32.778952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb6e98-3b64-4d80-9ac5-7758d305d6ca admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:32.842753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-86cb6e98-3b64-4d80-9ac5-7758d305d6ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dfdd7ea8c3a4c36a22b17049e7a1669', 'request_id': 'req-86cb6e98-3b64-4d80-9ac5-7758d305d6ca'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.855538 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-86cb6e98-3b64-4d80-9ac5-7758d305d6ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dfdd7ea8c3a4c36a22b17049e7a1669', 'request_id': 'req-86cb6e98-3b64-4d80-9ac5-7758d305d6ca'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.864602 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-86cb6e98-3b64-4d80-9ac5-7758d305d6ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dfdd7ea8c3a4c36a22b17049e7a1669', 'request_id': 'req-86cb6e98-3b64-4d80-9ac5-7758d305d6ca'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.878309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-86cb6e98-3b64-4d80-9ac5-7758d305d6ca admin admin] The token cache is being invalidate because project 3dfdd7ea8c3a4c36a22b17049e7a1669 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:52:32.878880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-86cb6e98-3b64-4d80-9ac5-7758d305d6ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.879799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-86cb6e98-3b64-4d80-9ac5-7758d305d6ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:32.882124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 828/1672] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:52:32 2022] DELETE /identity/v3/projects/3dfdd7ea8c3a4c36a22b17049e7a1669 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:32.883658 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfa6d5-8dc4-45ec-8ad0-c9ad06199c9d None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:32.885754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfa6d5-8dc4-45ec-8ad0-c9ad06199c9d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:32.886290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfa6d5-8dc4-45ec-8ad0-c9ad06199c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:32.907175 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-b2cfa6d5-8dc4-45ec-8ad0-c9ad06199c9d None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:32.909633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b2cfa6d5-8dc4-45ec-8ad0-c9ad06199c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:32.911670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4878b-a77f-4d99-8004-bfecd10ddf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:32.912403 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4878b-a77f-4d99-8004-bfecd10ddf7b admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:32.912695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4878b-a77f-4d99-8004-bfecd10ddf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:32.912971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4878b-a77f-4d99-8004-bfecd10ddf7b admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:32.913841 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4878b-a77f-4d99-8004-bfecd10ddf7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:52:32.914151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4878b-a77f-4d99-8004-bfecd10ddf7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:32.921233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4878b-a77f-4d99-8004-bfecd10ddf7b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:32.944788 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:32.961369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 845/1673] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:52:32.966189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4878b-a77f-4d99-8004-bfecd10ddf7b admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:32.973025 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-91d4878b-a77f-4d99-8004-bfecd10ddf7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:33.007381 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 829/1674] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:33.007707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4878b-a77f-4d99-8004-bfecd10ddf7b admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:33.009527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfa6d5-8dc4-45ec-8ad0-c9ad06199c9d None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:33.012413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b2cfa6d5-8dc4-45ec-8ad0-c9ad06199c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:33.018455 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c0c3e-7726-49d7-ac25-73b62f51d024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:33.019364 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c0c3e-7726-49d7-ac25-73b62f51d024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:33.019652 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c0c3e-7726-49d7-ac25-73b62f51d024 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:33.019960 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c0c3e-7726-49d7-ac25-73b62f51d024 admin admin] PATH_INFO: `/v3/users/44d9d1fce96746f6bebf01335d24d09f` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:33.023995 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c0c3e-7726-49d7-ac25-73b62f51d024 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44d9d1fce96746f6bebf01335d24d09f)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:33.030945 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfa6d5-8dc4-45ec-8ad0-c9ad06199c9d None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:33.033196 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c0c3e-7726-49d7-ac25-73b62f51d024 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:33.036960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab50c0-5305-4d3c-b89b-b2117b6cb7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:33.037950 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab50c0-5305-4d3c-b89b-b2117b6cb7bf admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:33.038578 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab50c0-5305-4d3c-b89b-b2117b6cb7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:33.039352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab50c0-5305-4d3c-b89b-b2117b6cb7bf admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:33.040410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab50c0-5305-4d3c-b89b-b2117b6cb7bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:33.044622 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab50c0-5305-4d3c-b89b-b2117b6cb7bf admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:33.125268 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 830/1675] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:33 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:33.126245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab50c0-5305-4d3c-b89b-b2117b6cb7bf admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:33.129848 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-03ab50c0-5305-4d3c-b89b-b2117b6cb7bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:33.136970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-640c0c3e-7726-49d7-ac25-73b62f51d024 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44d9d1fce96746f6bebf01335d24d09f'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:33.138406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-640c0c3e-7726-49d7-ac25-73b62f51d024 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44d9d1fce96746f6bebf01335d24d09f'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:33.176690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-640c0c3e-7726-49d7-ac25-73b62f51d024 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44d9d1fce96746f6bebf01335d24d09f'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:33.192620 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-640c0c3e-7726-49d7-ac25-73b62f51d024 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44d9d1fce96746f6bebf01335d24d09f'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:33.210153 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-640c0c3e-7726-49d7-ac25-73b62f51d024 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44d9d1fce96746f6bebf01335d24d09f'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:33.243901 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 846/1676] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:52:32 2022] DELETE /identity/v3/users/44d9d1fce96746f6bebf01335d24d09f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:33.247362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c0c3e-7726-49d7-ac25-73b62f51d024 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:33.249819 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-640c0c3e-7726-49d7-ac25-73b62f51d024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:33.292729 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab50c0-5305-4d3c-b89b-b2117b6cb7bf admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:33.366425 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318d21d-d5f8-425e-b587-2972dad34805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:33.369107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318d21d-d5f8-425e-b587-2972dad34805 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:33.370983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318d21d-d5f8-425e-b587-2972dad34805 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:33.371429 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318d21d-d5f8-425e-b587-2972dad34805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:33.374117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-2318d21d-d5f8-425e-b587-2972dad34805 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:33.428210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:33.522432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c0c3e-7726-49d7-ac25-73b62f51d024 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:33.533893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318d21d-d5f8-425e-b587-2972dad34805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:33.535382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318d21d-d5f8-425e-b587-2972dad34805 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:33.538891 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2abad-0e9b-4d9b-9aa0-b7bcfda59e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:33.539951 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2abad-0e9b-4d9b-9aa0-b7bcfda59e54 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:33.541929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2abad-0e9b-4d9b-9aa0-b7bcfda59e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:33.541929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2abad-0e9b-4d9b-9aa0-b7bcfda59e54 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:33.549271 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2abad-0e9b-4d9b-9aa0-b7bcfda59e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:33.552656 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 831/1677] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:33.556860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2abad-0e9b-4d9b-9aa0-b7bcfda59e54 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:33.622088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 847/1678] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:33 2022] POST /identity/v3/users => generated 290 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:33.627586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318d21d-d5f8-425e-b587-2972dad34805 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:33.645588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318d21d-d5f8-425e-b587-2972dad34805 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:33.657960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7198e-7286-490f-9416-c60546a2c087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:33.658820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7198e-7286-490f-9416-c60546a2c087 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:33.659119 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7198e-7286-490f-9416-c60546a2c087 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:33.659402 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7198e-7286-490f-9416-c60546a2c087 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:33.660259 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7198e-7286-490f-9416-c60546a2c087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:33.662648 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7198e-7286-490f-9416-c60546a2c087 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:33.671193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:33.678314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 832/1679] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:33.682537 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2abad-0e9b-4d9b-9aa0-b7bcfda59e54 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:33.694786 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2abad-0e9b-4d9b-9aa0-b7bcfda59e54 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:33.703421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696aa31-880b-4910-a47a-74f77e950022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:33.704562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696aa31-880b-4910-a47a-74f77e950022 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:33.705328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696aa31-880b-4910-a47a-74f77e950022 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:33.705779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696aa31-880b-4910-a47a-74f77e950022 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:33.706681 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696aa31-880b-4910-a47a-74f77e950022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:33.710978 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696aa31-880b-4910-a47a-74f77e950022 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:33.733603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 848/1680] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:33.738419 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7198e-7286-490f-9416-c60546a2c087 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:33.762266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7198e-7286-490f-9416-c60546a2c087 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:33.775509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2c685-e889-4c6f-a2db-bddbfea9cb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:33.776550 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2c685-e889-4c6f-a2db-bddbfea9cb3b admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:33.778638 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2c685-e889-4c6f-a2db-bddbfea9cb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:33.778638 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2c685-e889-4c6f-a2db-bddbfea9cb3b admin admin] PATH_INFO: `/v3/projects/bf7b8f3c6f4b4223b52dabcde2cc73b5/users/b9fff2fdf78b4e35b20980e577e1af9f/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:33.811743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2c685-e889-4c6f-a2db-bddbfea9cb3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf7b8f3c6f4b4223b52dabcde2cc73b5, user_id=b9fff2fdf78b4e35b20980e577e1af9f, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:33.816085 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2c685-e889-4c6f-a2db-bddbfea9cb3b admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:33.844239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 833/1681] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:33 2022] PUT /identity/v3/projects/bf7b8f3c6f4b4223b52dabcde2cc73b5/users/b9fff2fdf78b4e35b20980e577e1af9f/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:33.850082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dad7ba-7740-4602-9aaa-2c74d1de212b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:33.850643 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dad7ba-7740-4602-9aaa-2c74d1de212b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:33.851230 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dad7ba-7740-4602-9aaa-2c74d1de212b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:33.957434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-41dad7ba-7740-4602-9aaa-2c74d1de212b None None] MFA Rules not processed for user `b9fff2fdf78b4e35b20980e577e1af9f`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:33.963759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-41dad7ba-7740-4602-9aaa-2c74d1de212b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:34.150303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 849/1682] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:34.154887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2c685-e889-4c6f-a2db-bddbfea9cb3b admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:34.349877 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2c685-e889-4c6f-a2db-bddbfea9cb3b admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:34.369525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d23f0d-48b7-4dbc-8eb7-b279eca97634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:34.370739 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d23f0d-48b7-4dbc-8eb7-b279eca97634 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:34.371315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d23f0d-48b7-4dbc-8eb7-b279eca97634 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:34.371777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d23f0d-48b7-4dbc-8eb7-b279eca97634 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:34.373126 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d23f0d-48b7-4dbc-8eb7-b279eca97634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:34.377055 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d23f0d-48b7-4dbc-8eb7-b279eca97634 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:34.475498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 834/1683] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:34 2022] POST /identity/v3/projects => generated 379 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:34.480505 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dad7ba-7740-4602-9aaa-2c74d1de212b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:34.496011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dad7ba-7740-4602-9aaa-2c74d1de212b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:34.586367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1768c9-e157-4ba6-aa7e-8a1471a980b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:34.587003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1768c9-e157-4ba6-aa7e-8a1471a980b4 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:34.587355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1768c9-e157-4ba6-aa7e-8a1471a980b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:34.587641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1768c9-e157-4ba6-aa7e-8a1471a980b4 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:34.589105 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1768c9-e157-4ba6-aa7e-8a1471a980b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:34.591406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1768c9-e157-4ba6-aa7e-8a1471a980b4 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:34.621958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d23f0d-48b7-4dbc-8eb7-b279eca97634 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:34.622225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 850/1684] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:34 2022] POST /identity/v3/users => generated 296 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:34.642423 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d23f0d-48b7-4dbc-8eb7-b279eca97634 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:34.651830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02605c04-e9da-4da2-bda4-8b1aae0af8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:34.652879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02605c04-e9da-4da2-bda4-8b1aae0af8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:34.653313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02605c04-e9da-4da2-bda4-8b1aae0af8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:34.654590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02605c04-e9da-4da2-bda4-8b1aae0af8f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:34.656498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02605c04-e9da-4da2-bda4-8b1aae0af8f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:34.659580 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02605c04-e9da-4da2-bda4-8b1aae0af8f0 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:34.687373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1768c9-e157-4ba6-aa7e-8a1471a980b4 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:34.701295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 835/1685] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:52:34.704526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1768c9-e157-4ba6-aa7e-8a1471a980b4 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:34.712619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05662144-78b5-4ad0-97af-2f34d9f152b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:34.714762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05662144-78b5-4ad0-97af-2f34d9f152b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:34.715261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05662144-78b5-4ad0-97af-2f34d9f152b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:34.716519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05662144-78b5-4ad0-97af-2f34d9f152b1 admin admin] PATH_INFO: `/v3/projects/54ce6ee876cd4376a8bc1cda110d6298/users/6cd72c82e19942d2811996192bb8a2a9/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:34.752762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05662144-78b5-4ad0-97af-2f34d9f152b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54ce6ee876cd4376a8bc1cda110d6298, user_id=6cd72c82e19942d2811996192bb8a2a9, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:34.756265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05662144-78b5-4ad0-97af-2f34d9f152b1 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:34.799535 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 851/1686] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:34 2022] PUT /identity/v3/projects/54ce6ee876cd4376a8bc1cda110d6298/users/6cd72c82e19942d2811996192bb8a2a9/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:34.803612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02605c04-e9da-4da2-bda4-8b1aae0af8f0 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:34.815918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02605c04-e9da-4da2-bda4-8b1aae0af8f0 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:34.924253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7e71d-21db-4e62-a95a-b01db450f5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:34.925124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7e71d-21db-4e62-a95a-b01db450f5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:34.925808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7e71d-21db-4e62-a95a-b01db450f5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:34.926196 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7e71d-21db-4e62-a95a-b01db450f5b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:34.927209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7e71d-21db-4e62-a95a-b01db450f5b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:34.933220 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7e71d-21db-4e62-a95a-b01db450f5b0 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:34.955610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05662144-78b5-4ad0-97af-2f34d9f152b1 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:34.968608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 836/1687] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:34.976479 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05662144-78b5-4ad0-97af-2f34d9f152b1 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:34.982939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c576ab9-cf34-411e-8a7f-12e25226d25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:34.983901 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c576ab9-cf34-411e-8a7f-12e25226d25e admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:34.984192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c576ab9-cf34-411e-8a7f-12e25226d25e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:34.984473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c576ab9-cf34-411e-8a7f-12e25226d25e admin admin] PATH_INFO: `/v3/projects/54ce6ee876cd4376a8bc1cda110d6298/users/6cd72c82e19942d2811996192bb8a2a9/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:34.986815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c576ab9-cf34-411e-8a7f-12e25226d25e admin admin] RBAC: Authorizing `identity:create_grant(project_id=54ce6ee876cd4376a8bc1cda110d6298, user_id=6cd72c82e19942d2811996192bb8a2a9, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:34.989794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c576ab9-cf34-411e-8a7f-12e25226d25e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:35.003759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 852/1688] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:34 2022] PUT /identity/v3/projects/54ce6ee876cd4376a8bc1cda110d6298/users/6cd72c82e19942d2811996192bb8a2a9/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:35.025811 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790efb6-f304-4bb4-8b1c-2f0e860de1ba None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:35.026131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790efb6-f304-4bb4-8b1c-2f0e860de1ba None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:35.026382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790efb6-f304-4bb4-8b1c-2f0e860de1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:35.070981 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-5790efb6-f304-4bb4-8b1c-2f0e860de1ba None None] MFA Rules not processed for user `6cd72c82e19942d2811996192bb8a2a9`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:35.074214 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5790efb6-f304-4bb4-8b1c-2f0e860de1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:35.154504 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:35.254446 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c576ab9-cf34-411e-8a7f-12e25226d25e admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:35.256716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-7c576ab9-cf34-411e-8a7f-12e25226d25e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:35.264352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 837/1689] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:35.267530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790efb6-f304-4bb4-8b1c-2f0e860de1ba None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:35.269399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5790efb6-f304-4bb4-8b1c-2f0e860de1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:35.284860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:35.467747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790efb6-f304-4bb4-8b1c-2f0e860de1ba None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:35.471839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020ccd0-1d0b-4928-bbe6-05d525757a1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:35.472776 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020ccd0-1d0b-4928-bbe6-05d525757a1c service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:35.473037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020ccd0-1d0b-4928-bbe6-05d525757a1c service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:35.473327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020ccd0-1d0b-4928-bbe6-05d525757a1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:35.477737 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c576ab9-cf34-411e-8a7f-12e25226d25e admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:35.489753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020ccd0-1d0b-4928-bbe6-05d525757a1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:35.491875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020ccd0-1d0b-4928-bbe6-05d525757a1c service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:35.497700 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038593db-d9e3-4afe-93b3-bb77b7616d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:35.498613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038593db-d9e3-4afe-93b3-bb77b7616d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:35.498896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038593db-d9e3-4afe-93b3-bb77b7616d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:35.499172 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038593db-d9e3-4afe-93b3-bb77b7616d54 admin admin] PATH_INFO: `/v3/projects/8f7faeee165a4509a4c372e1c9de32cf` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:35.502833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038593db-d9e3-4afe-93b3-bb77b7616d54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f7faeee165a4509a4c372e1c9de32cf)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:35.504851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038593db-d9e3-4afe-93b3-bb77b7616d54 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:35.516515 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 838/1690] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:35.520598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc313e-49eb-47ac-b5a9-ee470cc1c6c0 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:35.521223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc313e-49eb-47ac-b5a9-ee470cc1c6c0 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:35.521520 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc313e-49eb-47ac-b5a9-ee470cc1c6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:35.612798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-edbc313e-49eb-47ac-b5a9-ee470cc1c6c0 None None] MFA Rules not processed for user `aba3ccf461484d9c84d3892eba8cef11`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:35.615248 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-edbc313e-49eb-47ac-b5a9-ee470cc1c6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:35.623589 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-038593db-d9e3-4afe-93b3-bb77b7616d54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f7faeee165a4509a4c372e1c9de32cf', 'request_id': 'req-038593db-d9e3-4afe-93b3-bb77b7616d54'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:35.631853 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-038593db-d9e3-4afe-93b3-bb77b7616d54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f7faeee165a4509a4c372e1c9de32cf', 'request_id': 'req-038593db-d9e3-4afe-93b3-bb77b7616d54'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:35.639440 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-038593db-d9e3-4afe-93b3-bb77b7616d54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f7faeee165a4509a4c372e1c9de32cf', 'request_id': 'req-038593db-d9e3-4afe-93b3-bb77b7616d54'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:35.662820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-038593db-d9e3-4afe-93b3-bb77b7616d54 admin admin] The token cache is being invalidate because project 8f7faeee165a4509a4c372e1c9de32cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:52:35.663201 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-038593db-d9e3-4afe-93b3-bb77b7616d54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:35.663993 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-038593db-d9e3-4afe-93b3-bb77b7616d54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:35.667898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 853/1691] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:52:35 2022] DELETE /identity/v3/projects/8f7faeee165a4509a4c372e1c9de32cf => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:35.667898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038593db-d9e3-4afe-93b3-bb77b7616d54 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:35.671643 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-038593db-d9e3-4afe-93b3-bb77b7616d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:35.783874 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 839/1692] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:35 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:35.785031 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc313e-49eb-47ac-b5a9-ee470cc1c6c0 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:35.789670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-edbc313e-49eb-47ac-b5a9-ee470cc1c6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:35.830144 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038593db-d9e3-4afe-93b3-bb77b7616d54 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:35.841293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc313e-49eb-47ac-b5a9-ee470cc1c6c0 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:35.842101 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ba92e-995a-42d4-84be-e214a419d5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:35.843224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ba92e-995a-42d4-84be-e214a419d5a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:35.843745 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ba92e-995a-42d4-84be-e214a419d5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:35.844311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ba92e-995a-42d4-84be-e214a419d5a0 admin admin] PATH_INFO: `/v3/projects/4b69684bfd6e47f2b6e5bb801b3140d8` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:35.847282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ba92e-995a-42d4-84be-e214a419d5a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b69684bfd6e47f2b6e5bb801b3140d8)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:35.848180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e7f10-418e-441d-9226-d383e1fc2c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:35.849193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e7f10-418e-441d-9226-d383e1fc2c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:35.849681 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e7f10-418e-441d-9226-d383e1fc2c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:35.850055 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e7f10-418e-441d-9226-d383e1fc2c82 admin admin] PATH_INFO: `/v3/projects/b221cf0ea3554c68946ad0014f2cd40a` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:35.850514 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ba92e-995a-42d4-84be-e214a419d5a0 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:35.851640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e7f10-418e-441d-9226-d383e1fc2c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b221cf0ea3554c68946ad0014f2cd40a)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:35.855887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e7f10-418e-441d-9226-d383e1fc2c82 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:35.972525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:35.981162 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9c1ba92e-995a-42d4-84be-e214a419d5a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b69684bfd6e47f2b6e5bb801b3140d8', 'request_id': 'req-9c1ba92e-995a-42d4-84be-e214a419d5a0'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:35.998643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-737e7f10-418e-441d-9226-d383e1fc2c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b221cf0ea3554c68946ad0014f2cd40a', 'request_id': 'req-737e7f10-418e-441d-9226-d383e1fc2c82'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:35.998901 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9c1ba92e-995a-42d4-84be-e214a419d5a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b69684bfd6e47f2b6e5bb801b3140d8', 'request_id': 'req-9c1ba92e-995a-42d4-84be-e214a419d5a0'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:36.004157 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:36.011482 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-737e7f10-418e-441d-9226-d383e1fc2c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b221cf0ea3554c68946ad0014f2cd40a', 'request_id': 'req-737e7f10-418e-441d-9226-d383e1fc2c82'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:36.011647 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9c1ba92e-995a-42d4-84be-e214a419d5a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b69684bfd6e47f2b6e5bb801b3140d8', 'request_id': 'req-9c1ba92e-995a-42d4-84be-e214a419d5a0'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:36.025265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-737e7f10-418e-441d-9226-d383e1fc2c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b221cf0ea3554c68946ad0014f2cd40a', 'request_id': 'req-737e7f10-418e-441d-9226-d383e1fc2c82'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:36.032628 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9c1ba92e-995a-42d4-84be-e214a419d5a0 admin admin] The token cache is being invalidate because project 4b69684bfd6e47f2b6e5bb801b3140d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:52:36.033257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9c1ba92e-995a-42d4-84be-e214a419d5a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:36.035284 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9c1ba92e-995a-42d4-84be-e214a419d5a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:36.044286 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 854/1693] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:52:35 2022] DELETE /identity/v3/projects/4b69684bfd6e47f2b6e5bb801b3140d8 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:36.045961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ba92e-995a-42d4-84be-e214a419d5a0 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:36.047470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9c1ba92e-995a-42d4-84be-e214a419d5a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:36.066049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-737e7f10-418e-441d-9226-d383e1fc2c82 admin admin] The token cache is being invalidate because project b221cf0ea3554c68946ad0014f2cd40a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:52:36.066353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-737e7f10-418e-441d-9226-d383e1fc2c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:36.067142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-737e7f10-418e-441d-9226-d383e1fc2c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:36.069038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 840/1694] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:52:35 2022] DELETE /identity/v3/projects/b221cf0ea3554c68946ad0014f2cd40a => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:36.070203 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a75bfe-076f-43b0-b736-9c1dbe469d81 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:36.070499 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a75bfe-076f-43b0-b736-9c1dbe469d81 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:36.070962 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a75bfe-076f-43b0-b736-9c1dbe469d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:36.117671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-b0a75bfe-076f-43b0-b736-9c1dbe469d81 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:36.119023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b0a75bfe-076f-43b0-b736-9c1dbe469d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:36.123114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ba92e-995a-42d4-84be-e214a419d5a0 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:36.198412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e1894-63ad-48b7-b48f-f047141bc0f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:36.203270 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e1894-63ad-48b7-b48f-f047141bc0f2 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:36.203566 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e1894-63ad-48b7-b48f-f047141bc0f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:36.203926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e1894-63ad-48b7-b48f-f047141bc0f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:36.208918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-719e1894-63ad-48b7-b48f-f047141bc0f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:36.243826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 841/1695] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:36.246138 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdca14b-24fd-4573-b3f9-6d9376b9b39a None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:36.252328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdca14b-24fd-4573-b3f9-6d9376b9b39a None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:36.252960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdca14b-24fd-4573-b3f9-6d9376b9b39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:36.283415 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e1894-63ad-48b7-b48f-f047141bc0f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:36.287639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e1894-63ad-48b7-b48f-f047141bc0f2 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:36.295501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-2bdca14b-24fd-4573-b3f9-6d9376b9b39a None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:36.296953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-2bdca14b-24fd-4573-b3f9-6d9376b9b39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:36.300419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 855/1696] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:36.301769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ebd01-a2e3-40ad-8b80-5c0ea52c8a01 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:36.302056 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ebd01-a2e3-40ad-8b80-5c0ea52c8a01 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:36.302330 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ebd01-a2e3-40ad-8b80-5c0ea52c8a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:36.336393 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 842/1697] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:36.338709 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3131c-ac22-415c-844d-658990970f33 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:36.349172 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3131c-ac22-415c-844d-658990970f33 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:36.350649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3131c-ac22-415c-844d-658990970f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:36.377195 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-3d2ebd01-a2e3-40ad-8b80-5c0ea52c8a01 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:36.380731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-3d2ebd01-a2e3-40ad-8b80-5c0ea52c8a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:36.409412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 856/1698] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:36.412076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c0b3131c-ac22-415c-844d-658990970f33 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:36.413089 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaaa5b-a0ec-46c3-a49b-3709cefa672d None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:36.413438 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaaa5b-a0ec-46c3-a49b-3709cefa672d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:36.413729 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaaa5b-a0ec-46c3-a49b-3709cefa672d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:36.414822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c0b3131c-ac22-415c-844d-658990970f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:36.444011 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 843/1699] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:36.444558 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdffb3-edb4-497e-a905-2cf10a763bf3 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:36.445234 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdffb3-edb4-497e-a905-2cf10a763bf3 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:36.445402 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdffb3-edb4-497e-a905-2cf10a763bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:36.461244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-efcaaa5b-a0ec-46c3-a49b-3709cefa672d None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:36.463355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-efcaaa5b-a0ec-46c3-a49b-3709cefa672d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:36.474359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 857/1700] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:36.476577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2afa5c-b162-4197-bb39-9755d6394935 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:36.477135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2afa5c-b162-4197-bb39-9755d6394935 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:36.477657 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2afa5c-b162-4197-bb39-9755d6394935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:36.480222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-8bbdffb3-edb4-497e-a905-2cf10a763bf3 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:36.481906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8bbdffb3-edb4-497e-a905-2cf10a763bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:36.499669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 844/1701] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:36.501687 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdffb3-edb4-497e-a905-2cf10a763bf3 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:36.518651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdffb3-edb4-497e-a905-2cf10a763bf3 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:36.526172 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6423116-a1b6-4ab9-8aae-b883b0284c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:36.527273 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6423116-a1b6-4ab9-8aae-b883b0284c1f admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:36.528118 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6423116-a1b6-4ab9-8aae-b883b0284c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:36.528669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6423116-a1b6-4ab9-8aae-b883b0284c1f admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:36.529776 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6423116-a1b6-4ab9-8aae-b883b0284c1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:52:36.530294 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6423116-a1b6-4ab9-8aae-b883b0284c1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:36.532748 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-ac2afa5c-b162-4197-bb39-9755d6394935 None None] MFA Rules not processed for user `bc6b0bfbda8944369d1a83b52b400f64`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:36.532890 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6423116-a1b6-4ab9-8aae-b883b0284c1f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:36.533992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ac2afa5c-b162-4197-bb39-9755d6394935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:36.549573 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 845/1702] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:52:36.551051 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6423116-a1b6-4ab9-8aae-b883b0284c1f admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:36.566391 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6423116-a1b6-4ab9-8aae-b883b0284c1f admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:36.574522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056c575-45d0-4af9-9fcb-5027863fddb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:36.575858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056c575-45d0-4af9-9fcb-5027863fddb1 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:36.576350 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056c575-45d0-4af9-9fcb-5027863fddb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:36.576826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056c575-45d0-4af9-9fcb-5027863fddb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:36.578552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:36.579903 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056c575-45d0-4af9-9fcb-5027863fddb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:52:36.580491 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056c575-45d0-4af9-9fcb-5027863fddb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:36.582750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056c575-45d0-4af9-9fcb-5027863fddb1 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:36.615016 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 846/1703] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:52:36.616292 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056c575-45d0-4af9-9fcb-5027863fddb1 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:36.631949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056c575-45d0-4af9-9fcb-5027863fddb1 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:36.641928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e975a-41e1-4280-b784-4e479f54e6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:36.645729 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e975a-41e1-4280-b784-4e479f54e6fb admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:36.646694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e975a-41e1-4280-b784-4e479f54e6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:36.646990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e975a-41e1-4280-b784-4e479f54e6fb admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:36.647799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e975a-41e1-4280-b784-4e479f54e6fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:52:36.648117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e975a-41e1-4280-b784-4e479f54e6fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:36.649766 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e975a-41e1-4280-b784-4e479f54e6fb admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:36.699677 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 847/1704] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:52:36.704543 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e975a-41e1-4280-b784-4e479f54e6fb admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:36.735046 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 858/1705] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:36.735046 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2afa5c-b162-4197-bb39-9755d6394935 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:36.742553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2afa5c-b162-4197-bb39-9755d6394935 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:36.743365 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e975a-41e1-4280-b784-4e479f54e6fb admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:36.751297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56904fb-4c5f-43ce-98b0-f329fba3d896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:36.752399 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56904fb-4c5f-43ce-98b0-f329fba3d896 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:36.753433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56904fb-4c5f-43ce-98b0-f329fba3d896 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:36.753781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56904fb-4c5f-43ce-98b0-f329fba3d896 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:36.755742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d086e-2bb4-4214-b8ba-e29719a1579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:36.757509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d086e-2bb4-4214-b8ba-e29719a1579e admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:36.758007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56904fb-4c5f-43ce-98b0-f329fba3d896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:36.758651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d086e-2bb4-4214-b8ba-e29719a1579e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:36.759581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d086e-2bb4-4214-b8ba-e29719a1579e admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:36.761689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d086e-2bb4-4214-b8ba-e29719a1579e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:36.762588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56904fb-4c5f-43ce-98b0-f329fba3d896 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:36.770087 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d086e-2bb4-4214-b8ba-e29719a1579e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:36.819158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 859/1706] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:36 2022] POST /identity/v3/projects => generated 357 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:36.821946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56904fb-4c5f-43ce-98b0-f329fba3d896 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:36.846810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 848/1707] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:36 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:36.849640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a88e1-be43-4caa-8328-7d3b8ea579e4 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:36.852303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a88e1-be43-4caa-8328-7d3b8ea579e4 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:36.854151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a88e1-be43-4caa-8328-7d3b8ea579e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:36.907454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-cc6a88e1-be43-4caa-8328-7d3b8ea579e4 None None] MFA Rules not processed for user `bc6b0bfbda8944369d1a83b52b400f64`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:36.914249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-cc6a88e1-be43-4caa-8328-7d3b8ea579e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:37.063911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56904fb-4c5f-43ce-98b0-f329fba3d896 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:37.111491 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 849/1708] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:37.117471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a88e1-be43-4caa-8328-7d3b8ea579e4 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:37.139798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:37.218011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de10fe-b78c-4521-a5e5-0453aa260236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:37.218882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de10fe-b78c-4521-a5e5-0453aa260236 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:37.220475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de10fe-b78c-4521-a5e5-0453aa260236 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:37.220872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de10fe-b78c-4521-a5e5-0453aa260236 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:37.221700 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de10fe-b78c-4521-a5e5-0453aa260236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:37.225351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de10fe-b78c-4521-a5e5-0453aa260236 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:37.293336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a88e1-be43-4caa-8328-7d3b8ea579e4 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:37.305370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 860/1709] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:36 2022] POST /identity/v3/projects => generated 409 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:37.306159 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de10fe-b78c-4521-a5e5-0453aa260236 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:37.312576 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c30c7c-94f6-4df6-854a-2937d1b3eceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:37.313301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c30c7c-94f6-4df6-854a-2937d1b3eceb admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:37.313591 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c30c7c-94f6-4df6-854a-2937d1b3eceb admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:37.313870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c30c7c-94f6-4df6-854a-2937d1b3eceb admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:37.314600 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c30c7c-94f6-4df6-854a-2937d1b3eceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:37.316643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c30c7c-94f6-4df6-854a-2937d1b3eceb admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:37.362287 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 850/1710] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:37 2022] POST /identity/v3/users => generated 279 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:37.366691 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c30c7c-94f6-4df6-854a-2937d1b3eceb admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:37.370501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-45c30c7c-94f6-4df6-854a-2937d1b3eceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:37.389476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:37.459797 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de10fe-b78c-4521-a5e5-0453aa260236 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:37.466197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebe573-7deb-460e-8b65-6772c170e06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:37.466907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebe573-7deb-460e-8b65-6772c170e06e admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:37.467251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebe573-7deb-460e-8b65-6772c170e06e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:37.467536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ebe573-7deb-460e-8b65-6772c170e06e admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:37.468809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ebe573-7deb-460e-8b65-6772c170e06e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:37.471272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ebe573-7deb-460e-8b65-6772c170e06e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:37.504866 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 861/1711] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:37 2022] POST /identity/v3/users => generated 283 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:37.506070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebe573-7deb-460e-8b65-6772c170e06e admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:37.519252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ebe573-7deb-460e-8b65-6772c170e06e admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:37.524411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb609c50-93c0-4c2d-a122-96c24673d68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:37.525531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb609c50-93c0-4c2d-a122-96c24673d68e admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:37.526305 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb609c50-93c0-4c2d-a122-96c24673d68e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:37.526756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb609c50-93c0-4c2d-a122-96c24673d68e admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:37.527479 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb609c50-93c0-4c2d-a122-96c24673d68e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:37.530293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb609c50-93c0-4c2d-a122-96c24673d68e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:37.540354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c30c7c-94f6-4df6-854a-2937d1b3eceb admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:37.562124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 862/1712] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:37 2022] POST /identity/v3/users => generated 305 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:37.562124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb609c50-93c0-4c2d-a122-96c24673d68e admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:37.583452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb609c50-93c0-4c2d-a122-96c24673d68e admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:37.601551 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f79553-7092-43c9-9fee-58ff5f25af76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:37.603088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f79553-7092-43c9-9fee-58ff5f25af76 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:37.604080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f79553-7092-43c9-9fee-58ff5f25af76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:37.604729 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f79553-7092-43c9-9fee-58ff5f25af76 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:37.606463 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f79553-7092-43c9-9fee-58ff5f25af76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:37.608611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f79553-7092-43c9-9fee-58ff5f25af76 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:37.627616 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 863/1713] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:37.628996 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f79553-7092-43c9-9fee-58ff5f25af76 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:37.642236 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f79553-7092-43c9-9fee-58ff5f25af76 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:37.648772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ad723-b2d6-4ccb-b647-2c517c9f1fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:37.649770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ad723-b2d6-4ccb-b647-2c517c9f1fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:37.650231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ad723-b2d6-4ccb-b647-2c517c9f1fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:37.651448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ad723-b2d6-4ccb-b647-2c517c9f1fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:37.652895 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ad723-b2d6-4ccb-b647-2c517c9f1fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:37.655167 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ad723-b2d6-4ccb-b647-2c517c9f1fd2 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:37.670758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 864/1714] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:37.672115 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ad723-b2d6-4ccb-b647-2c517c9f1fd2 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:37.676503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7073a-45e6-4f89-bc3e-a90cc1243b86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:37.677343 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7073a-45e6-4f89-bc3e-a90cc1243b86 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:37.677734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7073a-45e6-4f89-bc3e-a90cc1243b86 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:37.678150 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7073a-45e6-4f89-bc3e-a90cc1243b86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:37.691226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ad723-b2d6-4ccb-b647-2c517c9f1fd2 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:37.697654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e853964-adcc-4f75-a803-925d133dcd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:37.697654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e853964-adcc-4f75-a803-925d133dcd72 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:37.697654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e853964-adcc-4f75-a803-925d133dcd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:37.697966 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e853964-adcc-4f75-a803-925d133dcd72 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:37.698555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e853964-adcc-4f75-a803-925d133dcd72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:37.700808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e853964-adcc-4f75-a803-925d133dcd72 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:37.734631 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 865/1715] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:37.735590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e853964-adcc-4f75-a803-925d133dcd72 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:37.759537 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e853964-adcc-4f75-a803-925d133dcd72 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:37.771585 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:37.778187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403fd45-f233-4a17-910f-31dd50024ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:37.778893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403fd45-f233-4a17-910f-31dd50024ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:37.779349 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403fd45-f233-4a17-910f-31dd50024ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:37.779640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403fd45-f233-4a17-910f-31dd50024ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:37.780444 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403fd45-f233-4a17-910f-31dd50024ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:37.782932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403fd45-f233-4a17-910f-31dd50024ee4 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:37.812224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 866/1716] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:37.813598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403fd45-f233-4a17-910f-31dd50024ee4 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:37.841138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403fd45-f233-4a17-910f-31dd50024ee4 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:37.842658 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc7073a-45e6-4f89-bc3e-a90cc1243b86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:37.846847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc7073a-45e6-4f89-bc3e-a90cc1243b86 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:37.857921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0deec54-f60a-4fa2-9d21-4f1c84c400e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:37.859173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0deec54-f60a-4fa2-9d21-4f1c84c400e2 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:37.860603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0deec54-f60a-4fa2-9d21-4f1c84c400e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:37.861405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0deec54-f60a-4fa2-9d21-4f1c84c400e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:37.863644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0deec54-f60a-4fa2-9d21-4f1c84c400e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:37.866170 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0deec54-f60a-4fa2-9d21-4f1c84c400e2 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:37.876223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 851/1717] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:37.877253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7073a-45e6-4f89-bc3e-a90cc1243b86 service nova] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:37.884033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 867/1718] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:37.885623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0deec54-f60a-4fa2-9d21-4f1c84c400e2 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:37.898727 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0deec54-f60a-4fa2-9d21-4f1c84c400e2 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:37.905562 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7073a-45e6-4f89-bc3e-a90cc1243b86 service nova] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:37.912482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e0fa5-9f30-4e04-8ef7-3a02f91e3f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:37.917225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0fa5-9f30-4e04-8ef7-3a02f91e3f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:37.917452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0fa5-9f30-4e04-8ef7-3a02f91e3f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:37.917835 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0fa5-9f30-4e04-8ef7-3a02f91e3f29 admin admin] PATH_INFO: `/v3/projects/822e004d7a4245e0bba093ce0f7a3256/users/953ed636d6384f7489c2dec83864910b/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:37.918796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04b38a-eb72-4cda-a205-64180a855288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:37.919386 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04b38a-eb72-4cda-a205-64180a855288 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:37.919682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04b38a-eb72-4cda-a205-64180a855288 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:37.919994 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04b38a-eb72-4cda-a205-64180a855288 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:37.923211 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04b38a-eb72-4cda-a205-64180a855288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:37.926508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04b38a-eb72-4cda-a205-64180a855288 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:37.959347 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 852/1719] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:37.960483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04b38a-eb72-4cda-a205-64180a855288 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:37.977289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e0fa5-9f30-4e04-8ef7-3a02f91e3f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=822e004d7a4245e0bba093ce0f7a3256, user_id=953ed636d6384f7489c2dec83864910b, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:37.983789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e0fa5-9f30-4e04-8ef7-3a02f91e3f29 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:37.986234 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04b38a-eb72-4cda-a205-64180a855288 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:38.002775 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e604dfa-e955-4112-b90a-764b0d327cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:38.004245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e604dfa-e955-4112-b90a-764b0d327cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:38.004553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e604dfa-e955-4112-b90a-764b0d327cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:38.005556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e604dfa-e955-4112-b90a-764b0d327cc0 admin admin] PATH_INFO: `/v3/projects/dd56d36377314bd39f6a7f430cc20a5d/users/97da45de57674d8abbe1920d385a7f25/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:38.012942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 868/1720] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:37 2022] PUT /identity/v3/projects/822e004d7a4245e0bba093ce0f7a3256/users/953ed636d6384f7489c2dec83864910b/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:38.024363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e0fa5-9f30-4e04-8ef7-3a02f91e3f29 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:38.055880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e604dfa-e955-4112-b90a-764b0d327cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd56d36377314bd39f6a7f430cc20a5d, user_id=97da45de57674d8abbe1920d385a7f25, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:38.063119 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e604dfa-e955-4112-b90a-764b0d327cc0 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:38.105876 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 853/1721] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:37 2022] PUT /identity/v3/projects/dd56d36377314bd39f6a7f430cc20a5d/users/97da45de57674d8abbe1920d385a7f25/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:38.117044 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7c537-c387-4b70-a9a8-19dbcb5e4beb None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:38.117351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7c537-c387-4b70-a9a8-19dbcb5e4beb None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:38.117636 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7c537-c387-4b70-a9a8-19dbcb5e4beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:38.247033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c0d7c537-c387-4b70-a9a8-19dbcb5e4beb None None] MFA Rules not processed for user `953ed636d6384f7489c2dec83864910b`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:38.248711 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c0d7c537-c387-4b70-a9a8-19dbcb5e4beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:38.281361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e0fa5-9f30-4e04-8ef7-3a02f91e3f29 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:38.377168 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:38.467703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dc27e-e6da-4d2f-9ec8-5b979357b5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:38.468657 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dc27e-e6da-4d2f-9ec8-5b979357b5bb admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:38.468947 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dc27e-e6da-4d2f-9ec8-5b979357b5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:38.473211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dc27e-e6da-4d2f-9ec8-5b979357b5bb admin admin] PATH_INFO: `/v3/projects/58ebed32cfe044668a7501caf7af6ef8/users/c493c37862ea4e86b47cde6c1c487a0f/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:38.485672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 854/1722] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:38 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:38.488374 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4ce17-6759-4b7d-897b-41f1c8daa570 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:38.488858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4ce17-6759-4b7d-897b-41f1c8daa570 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:38.489295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4ce17-6759-4b7d-897b-41f1c8daa570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:38.527825 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dc27e-e6da-4d2f-9ec8-5b979357b5bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=58ebed32cfe044668a7501caf7af6ef8, user_id=c493c37862ea4e86b47cde6c1c487a0f, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:38.533009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dc27e-e6da-4d2f-9ec8-5b979357b5bb admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:38.543597 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-48e4ce17-6759-4b7d-897b-41f1c8daa570 None None] MFA Rules not processed for user `97da45de57674d8abbe1920d385a7f25`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:38.547904 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-48e4ce17-6759-4b7d-897b-41f1c8daa570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:38.565272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 869/1723] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:38 2022] PUT /identity/v3/projects/58ebed32cfe044668a7501caf7af6ef8/users/c493c37862ea4e86b47cde6c1c487a0f/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:38.568409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dc27e-e6da-4d2f-9ec8-5b979357b5bb admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:38.755575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dc27e-e6da-4d2f-9ec8-5b979357b5bb admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:38.767670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 855/1724] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:38.773447 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39052408-5665-4341-a1b6-d18ae535e2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:38.773948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951936cb-02c5-4ef9-832b-b3b34ddfa33c None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:38.774336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39052408-5665-4341-a1b6-d18ae535e2cb admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:38.774632 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39052408-5665-4341-a1b6-d18ae535e2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:38.774917 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39052408-5665-4341-a1b6-d18ae535e2cb admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:38.775841 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951936cb-02c5-4ef9-832b-b3b34ddfa33c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:38.778072 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951936cb-02c5-4ef9-832b-b3b34ddfa33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:38.784502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39052408-5665-4341-a1b6-d18ae535e2cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:38.791165 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39052408-5665-4341-a1b6-d18ae535e2cb admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:38.868020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-951936cb-02c5-4ef9-832b-b3b34ddfa33c None None] MFA Rules not processed for user `c493c37862ea4e86b47cde6c1c487a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:38.875337 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-951936cb-02c5-4ef9-832b-b3b34ddfa33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:38.879699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 870/1725] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:38 2022] POST /identity/v3/projects => generated 357 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:38.879699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991ce0d-91b0-4d4a-b57d-45aadeb9778f None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:38.879939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991ce0d-91b0-4d4a-b57d-45aadeb9778f None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:38.882909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991ce0d-91b0-4d4a-b57d-45aadeb9778f None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:38.882909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 871/1726] 10.222.0.60 () {54 vars in 921 bytes} [Thu Jul 21 20:52:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:52:38.882909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991ce0d-91b0-4d4a-b57d-45aadeb9778f None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:38.894405 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:39.102551 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991ce0d-91b0-4d4a-b57d-45aadeb9778f None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:39.108229 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7456ba-31ef-48ad-9e99-16684d4172f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:39.109339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7456ba-31ef-48ad-9e99-16684d4172f7 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:39.109944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7456ba-31ef-48ad-9e99-16684d4172f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:39.111323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7456ba-31ef-48ad-9e99-16684d4172f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:39.112313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7456ba-31ef-48ad-9e99-16684d4172f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:39.114986 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7456ba-31ef-48ad-9e99-16684d4172f7 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:39.185637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 872/1727] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:38 2022] POST /identity/v3/projects => generated 369 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:39.186449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7456ba-31ef-48ad-9e99-16684d4172f7 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:39.191389 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 856/1728] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:38 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:39.200661 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543350a0-7f03-492c-bbeb-0bac8a7825aa None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:39.200992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543350a0-7f03-492c-bbeb-0bac8a7825aa None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:39.201256 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543350a0-7f03-492c-bbeb-0bac8a7825aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:39.264423 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-543350a0-7f03-492c-bbeb-0bac8a7825aa None None] MFA Rules not processed for user `f4786051321f4a6f80a6cd23eacb09bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:39.265669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-543350a0-7f03-492c-bbeb-0bac8a7825aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:39.285212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:39.408194 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7456ba-31ef-48ad-9e99-16684d4172f7 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:39.424417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617427ef-c033-4a0e-9a38-9a2dc9c5af02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:39.428698 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617427ef-c033-4a0e-9a38-9a2dc9c5af02 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:39.437707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617427ef-c033-4a0e-9a38-9a2dc9c5af02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:39.441963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617427ef-c033-4a0e-9a38-9a2dc9c5af02 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:39.442817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617427ef-c033-4a0e-9a38-9a2dc9c5af02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:39.445663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617427ef-c033-4a0e-9a38-9a2dc9c5af02 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:39.447375 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 857/1729] 10.222.0.60 () {58 vars in 1018 bytes} [Thu Jul 21 20:52:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:39.449110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636c8ca-b633-4392-891e-57e1ffe744ef None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:39.449582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636c8ca-b633-4392-891e-57e1ffe744ef None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:39.450057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636c8ca-b633-4392-891e-57e1ffe744ef None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:39.458887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 858/1730] 10.222.0.60 () {54 vars in 912 bytes} [Thu Jul 21 20:52:39 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:52:39.462560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636c8ca-b633-4392-891e-57e1ffe744ef None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:39.493645 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636c8ca-b633-4392-891e-57e1ffe744ef None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:39.506363 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077985e-9c14-4858-b07f-25f109e76ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:39.507290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077985e-9c14-4858-b07f-25f109e76ece admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:39.507671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077985e-9c14-4858-b07f-25f109e76ece admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:39.508087 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077985e-9c14-4858-b07f-25f109e76ece admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:39.508897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077985e-9c14-4858-b07f-25f109e76ece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:39.511618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077985e-9c14-4858-b07f-25f109e76ece admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:39.537699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 873/1731] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:39 2022] POST /identity/v3/users => generated 285 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:39.538771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617427ef-c033-4a0e-9a38-9a2dc9c5af02 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:39.555844 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617427ef-c033-4a0e-9a38-9a2dc9c5af02 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:39.568116 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 859/1732] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:39 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:39.569761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977b8ae-1be7-4591-857a-105fa742fb26 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:39.570703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977b8ae-1be7-4591-857a-105fa742fb26 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:39.571562 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977b8ae-1be7-4591-857a-105fa742fb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:39.576733 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1cf60-9488-4c7b-9210-bc3313473681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:39.577670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1cf60-9488-4c7b-9210-bc3313473681 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:39.578636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1cf60-9488-4c7b-9210-bc3313473681 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:39.579124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1cf60-9488-4c7b-9210-bc3313473681 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:39.580814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1cf60-9488-4c7b-9210-bc3313473681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:39.583064 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1cf60-9488-4c7b-9210-bc3313473681 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:39.677867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 874/1733] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:39 2022] POST /identity/v3/projects => generated 407 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:39.692139 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31bf69-e59d-4e7f-a734-40507c8dff3c None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:39.692593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31bf69-e59d-4e7f-a734-40507c8dff3c None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:39.693050 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31bf69-e59d-4e7f-a734-40507c8dff3c None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:39.696208 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 875/1734] 10.222.0.60 () {54 vars in 961 bytes} [Thu Jul 21 20:52:39 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:52:39.697935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31bf69-e59d-4e7f-a734-40507c8dff3c None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:39.703048 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-4977b8ae-1be7-4591-857a-105fa742fb26 None None] MFA Rules not processed for user `33aff48933844401b86eb808937e35c4`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:39.704795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-4977b8ae-1be7-4591-857a-105fa742fb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:39.867317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 860/1735] 10.222.0.60 () {58 vars in 1009 bytes} [Thu Jul 21 20:52:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:39.868779 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977b8ae-1be7-4591-857a-105fa742fb26 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:39.906428 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977b8ae-1be7-4591-857a-105fa742fb26 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:39.907993 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31bf69-e59d-4e7f-a734-40507c8dff3c None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:39.919217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dff598-fae9-497d-9ce4-feadfdf44cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:39.920183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dff598-fae9-497d-9ce4-feadfdf44cfc admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:39.920576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dff598-fae9-497d-9ce4-feadfdf44cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:39.920956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dff598-fae9-497d-9ce4-feadfdf44cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:39.922162 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dff598-fae9-497d-9ce4-feadfdf44cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:39.923391 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1eb1e6-da75-43a7-be36-f47b77f144e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:39.924288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1eb1e6-da75-43a7-be36-f47b77f144e1 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:39.925835 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1eb1e6-da75-43a7-be36-f47b77f144e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:39.926523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dff598-fae9-497d-9ce4-feadfdf44cfc admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:39.929490 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1eb1e6-da75-43a7-be36-f47b77f144e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:39.932002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1eb1e6-da75-43a7-be36-f47b77f144e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:39.934711 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1eb1e6-da75-43a7-be36-f47b77f144e1 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:39.952265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 876/1736] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:39.959114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dff598-fae9-497d-9ce4-feadfdf44cfc admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:39.960677 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 861/1737] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:39.969959 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3345c-1796-4a5d-b41d-68e4197cb7ad None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:39.970354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3345c-1796-4a5d-b41d-68e4197cb7ad None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:39.970734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3345c-1796-4a5d-b41d-68e4197cb7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:39.984766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dff598-fae9-497d-9ce4-feadfdf44cfc admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:40.002093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba75a2-fa0c-4ad2-a5c4-a8c0e17b6dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:40.002948 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba75a2-fa0c-4ad2-a5c4-a8c0e17b6dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:40.003340 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba75a2-fa0c-4ad2-a5c4-a8c0e17b6dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:40.003731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba75a2-fa0c-4ad2-a5c4-a8c0e17b6dd6 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:40.004679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba75a2-fa0c-4ad2-a5c4-a8c0e17b6dd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:40.006899 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba75a2-fa0c-4ad2-a5c4-a8c0e17b6dd6 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:40.020126 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c6f3345c-1796-4a5d-b41d-68e4197cb7ad None None] MFA Rules not processed for user `f4786051321f4a6f80a6cd23eacb09bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:40.021971 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c6f3345c-1796-4a5d-b41d-68e4197cb7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:40.032631 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:40.055799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 877/1738] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:39 2022] POST /identity/v3/users => generated 310 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:40.056894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba75a2-fa0c-4ad2-a5c4-a8c0e17b6dd6 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:40.090290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba75a2-fa0c-4ad2-a5c4-a8c0e17b6dd6 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:40.119894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e8117-554f-4254-86c7-f682248e85ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:40.120994 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e8117-554f-4254-86c7-f682248e85ac admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:40.121393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e8117-554f-4254-86c7-f682248e85ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:40.121802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e8117-554f-4254-86c7-f682248e85ac admin admin] PATH_INFO: `/v3/projects/1213b68a36094fd9849498b886cc242f/users/f937daeb83e94358ac1bc9362ccc5ba8/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:40.189541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e8117-554f-4254-86c7-f682248e85ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=1213b68a36094fd9849498b886cc242f, user_id=f937daeb83e94358ac1bc9362ccc5ba8, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:40.194543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e8117-554f-4254-86c7-f682248e85ac admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:40.235444 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 878/1739] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:40 2022] PUT /identity/v3/projects/1213b68a36094fd9849498b886cc242f/users/f937daeb83e94358ac1bc9362ccc5ba8/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:40.237020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e8117-554f-4254-86c7-f682248e85ac admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:40.253815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 862/1740] 10.222.0.60 () {58 vars in 1058 bytes} [Thu Jul 21 20:52:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:40.254755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3345c-1796-4a5d-b41d-68e4197cb7ad None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:40.257849 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c6f3345c-1796-4a5d-b41d-68e4197cb7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:40.285913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3345c-1796-4a5d-b41d-68e4197cb7ad None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:40.317439 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:40.352366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e8117-554f-4254-86c7-f682248e85ac admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:40.369258 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdd58c-4db3-4486-a23e-e9d03370d742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:40.371475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdd58c-4db3-4486-a23e-e9d03370d742 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:40.371984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdd58c-4db3-4486-a23e-e9d03370d742 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:40.372377 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdd58c-4db3-4486-a23e-e9d03370d742 admin admin] PATH_INFO: `/v3/projects/8e1c8c4d767648c18d469117390435c0/users/3808aa2b8a674837b46505da8a3b1c6c/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:40.420531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfdd58c-4db3-4486-a23e-e9d03370d742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e1c8c4d767648c18d469117390435c0, user_id=3808aa2b8a674837b46505da8a3b1c6c, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:40.425774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfdd58c-4db3-4486-a23e-e9d03370d742 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:40.442467 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dded644-2b2e-4136-ac14-db4049ffb7a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:40.443205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dded644-2b2e-4136-ac14-db4049ffb7a2 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:40.443649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dded644-2b2e-4136-ac14-db4049ffb7a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:40.443979 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dded644-2b2e-4136-ac14-db4049ffb7a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:40.458331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 879/1741] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:40 2022] PUT /identity/v3/projects/8e1c8c4d767648c18d469117390435c0/users/3808aa2b8a674837b46505da8a3b1c6c/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:40.459389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdd58c-4db3-4486-a23e-e9d03370d742 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:40.475469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dded644-2b2e-4136-ac14-db4049ffb7a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:40.481779 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dded644-2b2e-4136-ac14-db4049ffb7a2 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:40.532313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 863/1742] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:40.533360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dded644-2b2e-4136-ac14-db4049ffb7a2 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:40.578427 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dded644-2b2e-4136-ac14-db4049ffb7a2 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:40.595301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdd58c-4db3-4486-a23e-e9d03370d742 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:40.599941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626018e9-e1a5-4a90-b87c-90e9b492de52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:40.601051 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626018e9-e1a5-4a90-b87c-90e9b492de52 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:40.601490 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626018e9-e1a5-4a90-b87c-90e9b492de52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:40.601898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626018e9-e1a5-4a90-b87c-90e9b492de52 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:40.608044 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626018e9-e1a5-4a90-b87c-90e9b492de52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:40.611592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626018e9-e1a5-4a90-b87c-90e9b492de52 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:40.617217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc0b50-a876-4d6d-97d3-40535843bffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:40.620708 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc0b50-a876-4d6d-97d3-40535843bffc admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:40.621005 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc0b50-a876-4d6d-97d3-40535843bffc admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:40.621291 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc0b50-a876-4d6d-97d3-40535843bffc admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:40.626790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc0b50-a876-4d6d-97d3-40535843bffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:40.631201 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc0b50-a876-4d6d-97d3-40535843bffc admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:40.643132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 864/1743] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:40.644977 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626018e9-e1a5-4a90-b87c-90e9b492de52 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:40.664892 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 880/1744] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:40.666832 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc0b50-a876-4d6d-97d3-40535843bffc admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:40.692553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc0b50-a876-4d6d-97d3-40535843bffc admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:40.704482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2ed52-dde7-4c11-be00-aa91f7b23d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:40.707886 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2ed52-dde7-4c11-be00-aa91f7b23d34 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:40.708183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2ed52-dde7-4c11-be00-aa91f7b23d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:40.708471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2ed52-dde7-4c11-be00-aa91f7b23d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:40.709402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2ed52-dde7-4c11-be00-aa91f7b23d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:40.717718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2ed52-dde7-4c11-be00-aa91f7b23d34 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:40.748287 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 881/1745] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:40.749183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2ed52-dde7-4c11-be00-aa91f7b23d34 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:40.763148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626018e9-e1a5-4a90-b87c-90e9b492de52 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:40.783182 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c054a-b9f8-4f12-b2eb-063317ba7220 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:40.784059 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c054a-b9f8-4f12-b2eb-063317ba7220 service placement] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:40.784836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c054a-b9f8-4f12-b2eb-063317ba7220 service placement] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:40.785178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c054a-b9f8-4f12-b2eb-063317ba7220 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:40.790991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2ed52-dde7-4c11-be00-aa91f7b23d34 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:40.799162 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c054a-b9f8-4f12-b2eb-063317ba7220 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:40.801040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c054a-b9f8-4f12-b2eb-063317ba7220 service placement] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:40.816728 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 865/1746] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:40.817669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c054a-b9f8-4f12-b2eb-063317ba7220 service placement] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:40.825917 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a9760-c1ae-40b8-9f24-1ddd4fd78517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:40.830521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a9760-c1ae-40b8-9f24-1ddd4fd78517 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:40.830814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a9760-c1ae-40b8-9f24-1ddd4fd78517 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:40.831104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a9760-c1ae-40b8-9f24-1ddd4fd78517 admin admin] PATH_INFO: `/v3/projects/1213b68a36094fd9849498b886cc242f/users/f937daeb83e94358ac1bc9362ccc5ba8/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:40.836994 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c054a-b9f8-4f12-b2eb-063317ba7220 service placement] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:40.846349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5580bf-6db2-4639-995a-6ffaf26906d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:40.846822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5580bf-6db2-4639-995a-6ffaf26906d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:40.847114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5580bf-6db2-4639-995a-6ffaf26906d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:40.848454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5580bf-6db2-4639-995a-6ffaf26906d2 admin admin] PATH_INFO: `/v3/projects/027b65cb8ab94c5eae06e15ef0e2f2a1/users/28e523eb8a774207a91d5630114f5cff/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:40.853969 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a9760-c1ae-40b8-9f24-1ddd4fd78517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1213b68a36094fd9849498b886cc242f, user_id=f937daeb83e94358ac1bc9362ccc5ba8, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:40.859040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a9760-c1ae-40b8-9f24-1ddd4fd78517 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:40.896184 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 882/1747] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:40 2022] PUT /identity/v3/projects/1213b68a36094fd9849498b886cc242f/users/f937daeb83e94358ac1bc9362ccc5ba8/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:40.897281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a9760-c1ae-40b8-9f24-1ddd4fd78517 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:40.913677 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5580bf-6db2-4639-995a-6ffaf26906d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=027b65cb8ab94c5eae06e15ef0e2f2a1, user_id=28e523eb8a774207a91d5630114f5cff, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:40.921800 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5580bf-6db2-4639-995a-6ffaf26906d2 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:40.958779 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 866/1748] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:40 2022] PUT /identity/v3/projects/027b65cb8ab94c5eae06e15ef0e2f2a1/users/28e523eb8a774207a91d5630114f5cff/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:40.961521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0db933-198e-4d67-9777-bcf3db149bd8 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:40.966505 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0db933-198e-4d67-9777-bcf3db149bd8 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:40.967587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0db933-198e-4d67-9777-bcf3db149bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:40.985008 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:41.020725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-9b0db933-198e-4d67-9777-bcf3db149bd8 None None] MFA Rules not processed for user `f937daeb83e94358ac1bc9362ccc5ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:41.023002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-9b0db933-198e-4d67-9777-bcf3db149bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:41.073661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a9760-c1ae-40b8-9f24-1ddd4fd78517 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:41.172861 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 867/1749] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:40 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:41.174145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0db933-198e-4d67-9777-bcf3db149bd8 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:41.208080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:41.247062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c208c7-d043-496f-955d-57135af6df70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:41.248076 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c208c7-d043-496f-955d-57135af6df70 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:41.248430 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c208c7-d043-496f-955d-57135af6df70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:41.248712 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c208c7-d043-496f-955d-57135af6df70 admin admin] PATH_INFO: `/v3/projects/8e1c8c4d767648c18d469117390435c0/users/3808aa2b8a674837b46505da8a3b1c6c/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:41.254215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c208c7-d043-496f-955d-57135af6df70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e1c8c4d767648c18d469117390435c0, user_id=3808aa2b8a674837b46505da8a3b1c6c, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:41.259207 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c208c7-d043-496f-955d-57135af6df70 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:41.305102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 883/1750] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:40 2022] PUT /identity/v3/projects/8e1c8c4d767648c18d469117390435c0/users/3808aa2b8a674837b46505da8a3b1c6c/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:41.306915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c208c7-d043-496f-955d-57135af6df70 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:41.310702 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-04c208c7-d043-496f-955d-57135af6df70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:41.344108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0db933-198e-4d67-9777-bcf3db149bd8 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:41.450009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c208c7-d043-496f-955d-57135af6df70 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:41.458438 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7baf2b-7a06-4fc9-8617-db23179a4d1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:41.459167 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7baf2b-7a06-4fc9-8617-db23179a4d1f service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:41.459452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7baf2b-7a06-4fc9-8617-db23179a4d1f service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:41.459792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7baf2b-7a06-4fc9-8617-db23179a4d1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:41.485615 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af991d-c898-49aa-9edd-5c41da37cdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:41.487975 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af991d-c898-49aa-9edd-5c41da37cdb9 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:41.488266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af991d-c898-49aa-9edd-5c41da37cdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:41.488549 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af991d-c898-49aa-9edd-5c41da37cdb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:41.490166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af991d-c898-49aa-9edd-5c41da37cdb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:41.492875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af991d-c898-49aa-9edd-5c41da37cdb9 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:41.516992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 868/1751] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:41.521033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93127049-d3ce-4e06-8eeb-fc868b5f2452 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:41.521192 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93127049-d3ce-4e06-8eeb-fc868b5f2452 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:41.521420 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93127049-d3ce-4e06-8eeb-fc868b5f2452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:41.578255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-93127049-d3ce-4e06-8eeb-fc868b5f2452 None None] MFA Rules not processed for user `3808aa2b8a674837b46505da8a3b1c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:41.580647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-93127049-d3ce-4e06-8eeb-fc868b5f2452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:41.603024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7baf2b-7a06-4fc9-8617-db23179a4d1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:41.605621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7baf2b-7a06-4fc9-8617-db23179a4d1f service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:41.632211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 884/1752] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:41 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:41.651357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:41.653971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7baf2b-7a06-4fc9-8617-db23179a4d1f service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:41.702287 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7baf2b-7a06-4fc9-8617-db23179a4d1f service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:41.720666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730625f-f384-4e96-bf77-85f4eedfae89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:41.721588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730625f-f384-4e96-bf77-85f4eedfae89 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:41.721882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730625f-f384-4e96-bf77-85f4eedfae89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:41.722165 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730625f-f384-4e96-bf77-85f4eedfae89 admin admin] PATH_INFO: `/v3/projects/027b65cb8ab94c5eae06e15ef0e2f2a1/users/28e523eb8a774207a91d5630114f5cff/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:41.731836 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730625f-f384-4e96-bf77-85f4eedfae89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=027b65cb8ab94c5eae06e15ef0e2f2a1, user_id=28e523eb8a774207a91d5630114f5cff, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:41.737941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730625f-f384-4e96-bf77-85f4eedfae89 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:41.768883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 885/1753] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:41 2022] PUT /identity/v3/projects/027b65cb8ab94c5eae06e15ef0e2f2a1/users/28e523eb8a774207a91d5630114f5cff/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:41.772175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e37e83-73fb-45f2-9ad2-8dd538788a74 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:41.772269 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e37e83-73fb-45f2-9ad2-8dd538788a74 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:41.772690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e37e83-73fb-45f2-9ad2-8dd538788a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:41.827244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-55e37e83-73fb-45f2-9ad2-8dd538788a74 None None] MFA Rules not processed for user `28e523eb8a774207a91d5630114f5cff`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:41.831912 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-55e37e83-73fb-45f2-9ad2-8dd538788a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:41.864454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 869/1754] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:41 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:41.865470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93127049-d3ce-4e06-8eeb-fc868b5f2452 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:41.886248 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93127049-d3ce-4e06-8eeb-fc868b5f2452 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:41.941849 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 886/1755] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:41 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:52:41.943232 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6c8f0-c235-4b0d-899c-64c700b59f68 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:41.943568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6c8f0-c235-4b0d-899c-64c700b59f68 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:41.943962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6c8f0-c235-4b0d-899c-64c700b59f68 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:41.945738 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 887/1756] 10.222.0.60 () {54 vars in 921 bytes} [Thu Jul 21 20:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:52:41.946506 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6c8f0-c235-4b0d-899c-64c700b59f68 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:41.952033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-65b6c8f0-c235-4b0d-899c-64c700b59f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:41.959783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:41.991004 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bcc66-d85f-4db5-8e20-37c9b19b20f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:41.992100 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bcc66-d85f-4db5-8e20-37c9b19b20f5 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:41.992892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bcc66-d85f-4db5-8e20-37c9b19b20f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:41.993346 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bcc66-d85f-4db5-8e20-37c9b19b20f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:42.071822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6c8f0-c235-4b0d-899c-64c700b59f68 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:42.082443 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f24bd-4305-4756-9528-7db7e471e132 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:42.083120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f24bd-4305-4756-9528-7db7e471e132 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:42.083408 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f24bd-4305-4756-9528-7db7e471e132 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:42.083690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f24bd-4305-4756-9528-7db7e471e132 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:42.109559 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bcc66-d85f-4db5-8e20-37c9b19b20f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:42.111683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bcc66-d85f-4db5-8e20-37c9b19b20f5 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:42.126680 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 870/1757] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:41 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:42.129661 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bb939-c140-469c-aa8e-55eaf5503a92 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:42.130153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bb939-c140-469c-aa8e-55eaf5503a92 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:42.130657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bb939-c140-469c-aa8e-55eaf5503a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:42.227458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-829bb939-c140-469c-aa8e-55eaf5503a92 None None] MFA Rules not processed for user `24e4efb87cd34008a1b83c0ea0cce5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:42.229296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-829bb939-c140-469c-aa8e-55eaf5503a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:42.261401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f24bd-4305-4756-9528-7db7e471e132 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:42.265363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f24bd-4305-4756-9528-7db7e471e132 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:42.303873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 888/1758] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:41 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:42.306927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8946d-60d8-42c5-8edf-0b562b51219b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:42.310963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8946d-60d8-42c5-8edf-0b562b51219b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:42.313355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8946d-60d8-42c5-8edf-0b562b51219b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:42.353641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-dec8946d-60d8-42c5-8edf-0b562b51219b None None] MFA Rules not processed for user `f4786051321f4a6f80a6cd23eacb09bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:42.355817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-dec8946d-60d8-42c5-8edf-0b562b51219b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:42.468367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:42.491089 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 871/1759] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:42.492342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bb939-c140-469c-aa8e-55eaf5503a92 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:42.494652 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-829bb939-c140-469c-aa8e-55eaf5503a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:42.513251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 889/1760] 10.222.0.60 () {58 vars in 1018 bytes} [Thu Jul 21 20:52:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:42.517560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8946d-60d8-42c5-8edf-0b562b51219b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:42.539745 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8946d-60d8-42c5-8edf-0b562b51219b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:42.560116 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242918f-cf14-428b-92e7-05166b524422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:42.561106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242918f-cf14-428b-92e7-05166b524422 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:42.561495 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242918f-cf14-428b-92e7-05166b524422 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:42.561880 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242918f-cf14-428b-92e7-05166b524422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:42.608511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242918f-cf14-428b-92e7-05166b524422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:42.609787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242918f-cf14-428b-92e7-05166b524422 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:42.642090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 890/1761] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:42 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:42.645565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbde381-21ea-429d-8e37-75bb8ba9fcde None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:42.645861 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbde381-21ea-429d-8e37-75bb8ba9fcde None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:42.646142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbde381-21ea-429d-8e37-75bb8ba9fcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:42.716905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-4dbde381-21ea-429d-8e37-75bb8ba9fcde None None] MFA Rules not processed for user `24e4efb87cd34008a1b83c0ea0cce5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:42.718680 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-4dbde381-21ea-429d-8e37-75bb8ba9fcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:42.755461 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 891/1762] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:42.756478 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbde381-21ea-429d-8e37-75bb8ba9fcde None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:42.796992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbde381-21ea-429d-8e37-75bb8ba9fcde None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:42.816586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699234a-4925-4d0f-bcdf-cf43509cd2d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:42.817272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699234a-4925-4d0f-bcdf-cf43509cd2d3 service placement] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:42.817853 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699234a-4925-4d0f-bcdf-cf43509cd2d3 service placement] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:42.818132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699234a-4925-4d0f-bcdf-cf43509cd2d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:42.837900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699234a-4925-4d0f-bcdf-cf43509cd2d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:42.839983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699234a-4925-4d0f-bcdf-cf43509cd2d3 service placement] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:42.866911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 892/1763] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:42.867832 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699234a-4925-4d0f-bcdf-cf43509cd2d3 service placement] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:42.874375 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bb939-c140-469c-aa8e-55eaf5503a92 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:42.913331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:42.921114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d0c18-0776-4724-b8bb-0ef289a24b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:42.939635 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d0c18-0776-4724-b8bb-0ef289a24b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:42.940341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d0c18-0776-4724-b8bb-0ef289a24b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:42.941102 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d0c18-0776-4724-b8bb-0ef289a24b27 admin admin] PATH_INFO: `/v3/users/abe04d8702cf47c788a9e309411c3273` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:42.943286 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d0c18-0776-4724-b8bb-0ef289a24b27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abe04d8702cf47c788a9e309411c3273)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:42.949675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d0c18-0776-4724-b8bb-0ef289a24b27 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:43.088663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699234a-4925-4d0f-bcdf-cf43509cd2d3 service placement] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:43.100795 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63924d6c-b79b-4a7d-8461-9457c354b9db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:43.101397 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63924d6c-b79b-4a7d-8461-9457c354b9db service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:43.101707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63924d6c-b79b-4a7d-8461-9457c354b9db service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:43.102027 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63924d6c-b79b-4a7d-8461-9457c354b9db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:43.136623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63924d6c-b79b-4a7d-8461-9457c354b9db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:43.138146 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63924d6c-b79b-4a7d-8461-9457c354b9db service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:43.175634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 893/1764] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:43.178355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63924d6c-b79b-4a7d-8461-9457c354b9db service nova] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:43.180930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-276d0c18-0776-4724-b8bb-0ef289a24b27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abe04d8702cf47c788a9e309411c3273'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:43.192044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-63924d6c-b79b-4a7d-8461-9457c354b9db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:43.199941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-276d0c18-0776-4724-b8bb-0ef289a24b27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abe04d8702cf47c788a9e309411c3273'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:43.201385 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-276d0c18-0776-4724-b8bb-0ef289a24b27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abe04d8702cf47c788a9e309411c3273'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:43.218628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-276d0c18-0776-4724-b8bb-0ef289a24b27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abe04d8702cf47c788a9e309411c3273'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:43.220776 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-276d0c18-0776-4724-b8bb-0ef289a24b27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abe04d8702cf47c788a9e309411c3273'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:43.257299 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63924d6c-b79b-4a7d-8461-9457c354b9db service nova] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:43.281483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453b095-be39-458e-a8c8-e6e80f25c610 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:43.282394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453b095-be39-458e-a8c8-e6e80f25c610 service placement] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:43.282778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453b095-be39-458e-a8c8-e6e80f25c610 service placement] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:43.283341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453b095-be39-458e-a8c8-e6e80f25c610 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:43.284787 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 872/1765] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:52:42 2022] DELETE /identity/v3/users/abe04d8702cf47c788a9e309411c3273 => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:43.285690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d0c18-0776-4724-b8bb-0ef289a24b27 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:43.286690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c453b095-be39-458e-a8c8-e6e80f25c610 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:43.289758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-276d0c18-0776-4724-b8bb-0ef289a24b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:43.302670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453b095-be39-458e-a8c8-e6e80f25c610 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:43.310355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453b095-be39-458e-a8c8-e6e80f25c610 service placement] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:43.344287 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 894/1766] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:43.425413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d0c18-0776-4724-b8bb-0ef289a24b27 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:43.431634 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771529b3-5738-44ae-9d55-fcb10c9cf670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:43.433500 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771529b3-5738-44ae-9d55-fcb10c9cf670 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:43.433500 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771529b3-5738-44ae-9d55-fcb10c9cf670 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:43.434249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771529b3-5738-44ae-9d55-fcb10c9cf670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:43.436330 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-771529b3-5738-44ae-9d55-fcb10c9cf670 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:43.522895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:43.606824 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776459c-3ea1-4b6f-bca1-d2a852f16de9 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:43.607661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776459c-3ea1-4b6f-bca1-d2a852f16de9 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:43.609592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776459c-3ea1-4b6f-bca1-d2a852f16de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:43.622913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771529b3-5738-44ae-9d55-fcb10c9cf670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:43.626157 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771529b3-5738-44ae-9d55-fcb10c9cf670 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:43.662732 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 873/1767] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:43 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:43.699295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-f776459c-3ea1-4b6f-bca1-d2a852f16de9 None None] MFA Rules not processed for user `3a0d900f08d54098870ebbfa281e2a65`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:43.700857 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f776459c-3ea1-4b6f-bca1-d2a852f16de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:43.900868 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 895/1768] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:43.920680 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771529b3-5738-44ae-9d55-fcb10c9cf670 service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:43.924630 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-771529b3-5738-44ae-9d55-fcb10c9cf670 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:43.940265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776459c-3ea1-4b6f-bca1-d2a852f16de9 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:43.942864 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f776459c-3ea1-4b6f-bca1-d2a852f16de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:44.202359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776459c-3ea1-4b6f-bca1-d2a852f16de9 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:44.218290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925116f1-87c9-4400-80bb-e4bcfa4f1b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:44.219843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771529b3-5738-44ae-9d55-fcb10c9cf670 service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:44.230388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a40444-a164-4a65-a92b-6b92cc2f2315 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:44.231505 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a40444-a164-4a65-a92b-6b92cc2f2315 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:44.232192 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a40444-a164-4a65-a92b-6b92cc2f2315 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:44.232992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a40444-a164-4a65-a92b-6b92cc2f2315 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:44.247237 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a40444-a164-4a65-a92b-6b92cc2f2315 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:44.253849 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925116f1-87c9-4400-80bb-e4bcfa4f1b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:44.254276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925116f1-87c9-4400-80bb-e4bcfa4f1b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:44.254380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a40444-a164-4a65-a92b-6b92cc2f2315 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:44.254752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925116f1-87c9-4400-80bb-e4bcfa4f1b39 admin admin] PATH_INFO: `/v3/projects/1e9e2fb1afd34141bb9cb2d2de1694ab` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:44.277122 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925116f1-87c9-4400-80bb-e4bcfa4f1b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e9e2fb1afd34141bb9cb2d2de1694ab)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:44.277944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925116f1-87c9-4400-80bb-e4bcfa4f1b39 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:44.286887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 874/1769] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:44.420685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:44.533246 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-925116f1-87c9-4400-80bb-e4bcfa4f1b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e9e2fb1afd34141bb9cb2d2de1694ab', 'request_id': 'req-925116f1-87c9-4400-80bb-e4bcfa4f1b39'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:44.549893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-925116f1-87c9-4400-80bb-e4bcfa4f1b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e9e2fb1afd34141bb9cb2d2de1694ab', 'request_id': 'req-925116f1-87c9-4400-80bb-e4bcfa4f1b39'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:44.568384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-925116f1-87c9-4400-80bb-e4bcfa4f1b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e9e2fb1afd34141bb9cb2d2de1694ab', 'request_id': 'req-925116f1-87c9-4400-80bb-e4bcfa4f1b39'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:44.615425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-925116f1-87c9-4400-80bb-e4bcfa4f1b39 admin admin] The token cache is being invalidate because project 1e9e2fb1afd34141bb9cb2d2de1694ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:52:44.616136 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-925116f1-87c9-4400-80bb-e4bcfa4f1b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:44.624987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-925116f1-87c9-4400-80bb-e4bcfa4f1b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:44.627782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 896/1770] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:52:43 2022] DELETE /identity/v3/projects/1e9e2fb1afd34141bb9cb2d2de1694ab => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:44.631793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a40444-a164-4a65-a92b-6b92cc2f2315 service nova] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:44.655888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a40444-a164-4a65-a92b-6b92cc2f2315 service nova] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:44.869036 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4186b-837d-4e6b-a7b0-0bf3b2e3ff31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:44.870286 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4186b-837d-4e6b-a7b0-0bf3b2e3ff31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:44.870908 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4186b-837d-4e6b-a7b0-0bf3b2e3ff31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:44.871356 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4186b-837d-4e6b-a7b0-0bf3b2e3ff31 admin admin] PATH_INFO: `/v3/users/aba3ccf461484d9c84d3892eba8cef11` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:44.876937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4186b-837d-4e6b-a7b0-0bf3b2e3ff31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aba3ccf461484d9c84d3892eba8cef11)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:44.880177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4186b-837d-4e6b-a7b0-0bf3b2e3ff31 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:44.982888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-26e4186b-837d-4e6b-a7b0-0bf3b2e3ff31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aba3ccf461484d9c84d3892eba8cef11'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:44.998375 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-26e4186b-837d-4e6b-a7b0-0bf3b2e3ff31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aba3ccf461484d9c84d3892eba8cef11'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:44.999704 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-26e4186b-837d-4e6b-a7b0-0bf3b2e3ff31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aba3ccf461484d9c84d3892eba8cef11'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:45.007171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:45.016324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-26e4186b-837d-4e6b-a7b0-0bf3b2e3ff31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aba3ccf461484d9c84d3892eba8cef11'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:45.017233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-26e4186b-837d-4e6b-a7b0-0bf3b2e3ff31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aba3ccf461484d9c84d3892eba8cef11'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:45.078231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 875/1771] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:52:44 2022] DELETE /identity/v3/users/aba3ccf461484d9c84d3892eba8cef11 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:45.217587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2202c-fb41-45af-8559-b0f1913bae59 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:45.217884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2202c-fb41-45af-8559-b0f1913bae59 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:45.218172 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2202c-fb41-45af-8559-b0f1913bae59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:45.266104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-eca2202c-fb41-45af-8559-b0f1913bae59 None None] MFA Rules not processed for user `24e4efb87cd34008a1b83c0ea0cce5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:45.280483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-eca2202c-fb41-45af-8559-b0f1913bae59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:45.489813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 897/1772] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:45.749796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4186b-837d-4e6b-a7b0-0bf3b2e3ff31 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:45.753978 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-26e4186b-837d-4e6b-a7b0-0bf3b2e3ff31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:45.986492 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4186b-837d-4e6b-a7b0-0bf3b2e3ff31 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:46.002307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608089e-7536-4836-bce8-21960f2f4f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:46.003517 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608089e-7536-4836-bce8-21960f2f4f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:46.004118 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608089e-7536-4836-bce8-21960f2f4f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:46.004703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608089e-7536-4836-bce8-21960f2f4f2e admin admin] PATH_INFO: `/v3/projects/09f927548fe047218578d81e7cc92f8b` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:46.009610 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608089e-7536-4836-bce8-21960f2f4f2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=09f927548fe047218578d81e7cc92f8b)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:46.013022 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608089e-7536-4836-bce8-21960f2f4f2e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:46.017388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2202c-fb41-45af-8559-b0f1913bae59 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:46.020294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-eca2202c-fb41-45af-8559-b0f1913bae59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:46.049942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2202c-fb41-45af-8559-b0f1913bae59 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:46.069653 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff27de-6aa0-4b10-b369-61ca9ef06498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:46.070559 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff27de-6aa0-4b10-b369-61ca9ef06498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:46.070840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff27de-6aa0-4b10-b369-61ca9ef06498 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:46.071116 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff27de-6aa0-4b10-b369-61ca9ef06498 admin admin] PATH_INFO: `/v3/users/97da45de57674d8abbe1920d385a7f25` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:46.073025 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff27de-6aa0-4b10-b369-61ca9ef06498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97da45de57674d8abbe1920d385a7f25)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:46.074829 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff27de-6aa0-4b10-b369-61ca9ef06498 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:46.124930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b608089e-7536-4836-bce8-21960f2f4f2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09f927548fe047218578d81e7cc92f8b', 'request_id': 'req-b608089e-7536-4836-bce8-21960f2f4f2e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:46.154316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b608089e-7536-4836-bce8-21960f2f4f2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09f927548fe047218578d81e7cc92f8b', 'request_id': 'req-b608089e-7536-4836-bce8-21960f2f4f2e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:46.176062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b608089e-7536-4836-bce8-21960f2f4f2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09f927548fe047218578d81e7cc92f8b', 'request_id': 'req-b608089e-7536-4836-bce8-21960f2f4f2e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:46.185341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-24ff27de-6aa0-4b10-b369-61ca9ef06498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97da45de57674d8abbe1920d385a7f25'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:46.186199 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-24ff27de-6aa0-4b10-b369-61ca9ef06498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97da45de57674d8abbe1920d385a7f25'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:46.220208 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-24ff27de-6aa0-4b10-b369-61ca9ef06498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97da45de57674d8abbe1920d385a7f25'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:46.220633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b608089e-7536-4836-bce8-21960f2f4f2e admin admin] The token cache is being invalidate because project 09f927548fe047218578d81e7cc92f8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:52:46.220633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b608089e-7536-4836-bce8-21960f2f4f2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:46.220633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b608089e-7536-4836-bce8-21960f2f4f2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:46.222063 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 876/1773] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:52:45 2022] DELETE /identity/v3/projects/09f927548fe047218578d81e7cc92f8b => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:46.222673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b281403-6929-4c82-b570-7de512e62f17 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:46.222965 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b281403-6929-4c82-b570-7de512e62f17 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:46.225136 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b281403-6929-4c82-b570-7de512e62f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:46.227774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-24ff27de-6aa0-4b10-b369-61ca9ef06498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97da45de57674d8abbe1920d385a7f25'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:46.242253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-24ff27de-6aa0-4b10-b369-61ca9ef06498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97da45de57674d8abbe1920d385a7f25'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:46.250506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-1b281403-6929-4c82-b570-7de512e62f17 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:46.252916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1b281403-6929-4c82-b570-7de512e62f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:46.253998 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 898/1774] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:52:46 2022] DELETE /identity/v3/users/97da45de57674d8abbe1920d385a7f25 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:52:46.258958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320ec95-f959-478b-86b3-fb2cda14e874 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:46.259265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320ec95-f959-478b-86b3-fb2cda14e874 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:46.259604 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320ec95-f959-478b-86b3-fb2cda14e874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:46.301480 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-2320ec95-f959-478b-86b3-fb2cda14e874 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:46.304196 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-2320ec95-f959-478b-86b3-fb2cda14e874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:46.349544 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:46.415378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 877/1775] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:46.419852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e270df-073e-404f-aa0d-02ab662867a5 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:46.419852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e270df-073e-404f-aa0d-02ab662867a5 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:46.419852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e270df-073e-404f-aa0d-02ab662867a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:46.443158 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-42e270df-073e-404f-aa0d-02ab662867a5 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:46.445286 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-42e270df-073e-404f-aa0d-02ab662867a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:46.449551 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 899/1776] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:46.457622 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320ec95-f959-478b-86b3-fb2cda14e874 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:46.459998 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-2320ec95-f959-478b-86b3-fb2cda14e874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:46.482290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 878/1777] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:46.484916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e270df-073e-404f-aa0d-02ab662867a5 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:46.496295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:46.515939 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e270df-073e-404f-aa0d-02ab662867a5 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:46.524376 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0331b9-30fd-40a2-b3c5-bf03d418a38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:46.526027 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0331b9-30fd-40a2-b3c5-bf03d418a38a admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:46.526627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0331b9-30fd-40a2-b3c5-bf03d418a38a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:46.527131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0331b9-30fd-40a2-b3c5-bf03d418a38a admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:46.531075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0331b9-30fd-40a2-b3c5-bf03d418a38a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:52:46.531766 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0331b9-30fd-40a2-b3c5-bf03d418a38a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:46.533955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0331b9-30fd-40a2-b3c5-bf03d418a38a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:46.574282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 879/1778] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:52:46.575430 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0331b9-30fd-40a2-b3c5-bf03d418a38a admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:46.586899 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320ec95-f959-478b-86b3-fb2cda14e874 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:46.590400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0331b9-30fd-40a2-b3c5-bf03d418a38a admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:46.594293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0b2eb-bb11-473b-84de-52f962a0ea0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:46.596565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0b2eb-bb11-473b-84de-52f962a0ea0a service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:46.596997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0b2eb-bb11-473b-84de-52f962a0ea0a service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:46.597505 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0b2eb-bb11-473b-84de-52f962a0ea0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:46.599093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a0c78-6096-49e4-9b3e-a84b1f63e0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:46.600066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a0c78-6096-49e4-9b3e-a84b1f63e0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:46.600551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a0c78-6096-49e4-9b3e-a84b1f63e0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:46.601154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a0c78-6096-49e4-9b3e-a84b1f63e0d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:46.602199 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a0c78-6096-49e4-9b3e-a84b1f63e0d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:46.611650 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a0c78-6096-49e4-9b3e-a84b1f63e0d6 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:46.623033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0b2eb-bb11-473b-84de-52f962a0ea0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:46.626622 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0b2eb-bb11-473b-84de-52f962a0ea0a service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:46.641362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 900/1779] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:46.655799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 880/1780] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:46 2022] POST /identity/v3/projects => generated 365 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:46.661274 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0b2eb-bb11-473b-84de-52f962a0ea0a service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:46.678359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0b2eb-bb11-473b-84de-52f962a0ea0a service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:46.838704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bc796-4b9b-406f-ab2d-111de8be4ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:46.839646 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bc796-4b9b-406f-ab2d-111de8be4ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:46.839961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bc796-4b9b-406f-ab2d-111de8be4ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:46.840245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bc796-4b9b-406f-ab2d-111de8be4ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:46.840963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bc796-4b9b-406f-ab2d-111de8be4ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:46.842927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bc796-4b9b-406f-ab2d-111de8be4ee3 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:46.925413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a0c78-6096-49e4-9b3e-a84b1f63e0d6 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:46.934256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 901/1781] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:46 2022] POST /identity/v3/users => generated 283 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:46.977234 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a0c78-6096-49e4-9b3e-a84b1f63e0d6 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:46.985543 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5e7cc-f84a-444a-98ac-9af1d1d4d64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:46.988372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5e7cc-f84a-444a-98ac-9af1d1d4d64b admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:46.990130 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5e7cc-f84a-444a-98ac-9af1d1d4d64b admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:46.990988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5e7cc-f84a-444a-98ac-9af1d1d4d64b admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:46.998960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5e7cc-f84a-444a-98ac-9af1d1d4d64b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:47.001147 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5e7cc-f84a-444a-98ac-9af1d1d4d64b admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:47.018672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bc796-4b9b-406f-ab2d-111de8be4ee3 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:47.027854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 881/1782] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:47.032286 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bc796-4b9b-406f-ab2d-111de8be4ee3 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:47.038168 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e9956-9aaa-4f30-acab-6fab01e03ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:47.039133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e9956-9aaa-4f30-acab-6fab01e03ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:47.040241 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e9956-9aaa-4f30-acab-6fab01e03ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:47.041611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e9956-9aaa-4f30-acab-6fab01e03ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:47.043038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e9956-9aaa-4f30-acab-6fab01e03ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:47.044900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e9956-9aaa-4f30-acab-6fab01e03ff9 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:47.067171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5e7cc-f84a-444a-98ac-9af1d1d4d64b admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:47.079808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 902/1783] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:47.084729 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5e7cc-f84a-444a-98ac-9af1d1d4d64b admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:47.091977 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb79ee-fd63-4048-bb06-889fb572f492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:47.093043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb79ee-fd63-4048-bb06-889fb572f492 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:47.093742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb79ee-fd63-4048-bb06-889fb572f492 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:47.093742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb79ee-fd63-4048-bb06-889fb572f492 admin admin] PATH_INFO: `/v3/projects/a84fe5a2425c43f581bf505c96f43ee3/users/bc6344e800394e0c8d19318b76d7d650/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:47.115214 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb79ee-fd63-4048-bb06-889fb572f492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84fe5a2425c43f581bf505c96f43ee3, user_id=bc6344e800394e0c8d19318b76d7d650, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:47.117881 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb79ee-fd63-4048-bb06-889fb572f492 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:47.148309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 882/1784] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:47 2022] PUT /identity/v3/projects/a84fe5a2425c43f581bf505c96f43ee3/users/bc6344e800394e0c8d19318b76d7d650/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:47.158326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922dee0-8cd9-4b85-8ae5-17538c0425db None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:47.158622 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922dee0-8cd9-4b85-8ae5-17538c0425db None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:47.158910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922dee0-8cd9-4b85-8ae5-17538c0425db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:47.202120 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb79ee-fd63-4048-bb06-889fb572f492 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:47.204358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-bcbb79ee-fd63-4048-bb06-889fb572f492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:47.258665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-c922dee0-8cd9-4b85-8ae5-17538c0425db None None] MFA Rules not processed for user `bc6344e800394e0c8d19318b76d7d650`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:47.281860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c922dee0-8cd9-4b85-8ae5-17538c0425db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:47.405368 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb79ee-fd63-4048-bb06-889fb572f492 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:47.410993 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:47.412897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f80284-9be7-4bc8-bb62-8f7b257b7a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:47.414258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f80284-9be7-4bc8-bb62-8f7b257b7a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:47.414779 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f80284-9be7-4bc8-bb62-8f7b257b7a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:47.415361 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f80284-9be7-4bc8-bb62-8f7b257b7a17 admin admin] PATH_INFO: `/v3/projects/dd56d36377314bd39f6a7f430cc20a5d` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:47.420658 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f80284-9be7-4bc8-bb62-8f7b257b7a17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd56d36377314bd39f6a7f430cc20a5d)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:47.424942 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f80284-9be7-4bc8-bb62-8f7b257b7a17 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:47.460803 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 903/1785] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:47.468221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922dee0-8cd9-4b85-8ae5-17538c0425db None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:47.509503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922dee0-8cd9-4b85-8ae5-17538c0425db None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:47.563972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-48f80284-9be7-4bc8-bb62-8f7b257b7a17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd56d36377314bd39f6a7f430cc20a5d', 'request_id': 'req-48f80284-9be7-4bc8-bb62-8f7b257b7a17'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:47.592263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-48f80284-9be7-4bc8-bb62-8f7b257b7a17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd56d36377314bd39f6a7f430cc20a5d', 'request_id': 'req-48f80284-9be7-4bc8-bb62-8f7b257b7a17'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:47.627367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-48f80284-9be7-4bc8-bb62-8f7b257b7a17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd56d36377314bd39f6a7f430cc20a5d', 'request_id': 'req-48f80284-9be7-4bc8-bb62-8f7b257b7a17'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:47.656154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc409ac6-c729-4d2e-a6e6-4fb66368b839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:47.657314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc409ac6-c729-4d2e-a6e6-4fb66368b839 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:47.657785 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc409ac6-c729-4d2e-a6e6-4fb66368b839 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:47.658106 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-48f80284-9be7-4bc8-bb62-8f7b257b7a17 admin admin] The token cache is being invalidate because project dd56d36377314bd39f6a7f430cc20a5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:52:47.658434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc409ac6-c729-4d2e-a6e6-4fb66368b839 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:47.658559 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-48f80284-9be7-4bc8-bb62-8f7b257b7a17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:47.660465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-48f80284-9be7-4bc8-bb62-8f7b257b7a17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:47.661199 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc409ac6-c729-4d2e-a6e6-4fb66368b839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:47.667497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc409ac6-c729-4d2e-a6e6-4fb66368b839 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:47.670264 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 883/1786] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:52:47 2022] DELETE /identity/v3/projects/dd56d36377314bd39f6a7f430cc20a5d => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:52:47.671431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f80284-9be7-4bc8-bb62-8f7b257b7a17 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:47.673969 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-48f80284-9be7-4bc8-bb62-8f7b257b7a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:47.697606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f80284-9be7-4bc8-bb62-8f7b257b7a17 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:47.738883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404ea77-d49c-416d-9faf-292ca2361d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:47.742135 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404ea77-d49c-416d-9faf-292ca2361d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:47.743015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404ea77-d49c-416d-9faf-292ca2361d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:47.744601 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404ea77-d49c-416d-9faf-292ca2361d0b admin admin] PATH_INFO: `/v3/users/3808aa2b8a674837b46505da8a3b1c6c` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:47.750856 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 904/1787] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:47 2022] POST /identity/v3/projects => generated 363 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:47.754328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc409ac6-c729-4d2e-a6e6-4fb66368b839 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:47.755875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404ea77-d49c-416d-9faf-292ca2361d0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3808aa2b8a674837b46505da8a3b1c6c)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:47.760507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404ea77-d49c-416d-9faf-292ca2361d0b admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:47.762139 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fc409ac6-c729-4d2e-a6e6-4fb66368b839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:47.810473 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:47.871400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0404ea77-d49c-416d-9faf-292ca2361d0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3808aa2b8a674837b46505da8a3b1c6c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:47.885217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0404ea77-d49c-416d-9faf-292ca2361d0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3808aa2b8a674837b46505da8a3b1c6c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:47.885988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0404ea77-d49c-416d-9faf-292ca2361d0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3808aa2b8a674837b46505da8a3b1c6c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:47.901480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0404ea77-d49c-416d-9faf-292ca2361d0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3808aa2b8a674837b46505da8a3b1c6c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:47.902982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0404ea77-d49c-416d-9faf-292ca2361d0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3808aa2b8a674837b46505da8a3b1c6c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:47.918527 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc409ac6-c729-4d2e-a6e6-4fb66368b839 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:47.942791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6d3f6-28c6-454b-b5b8-4a3fe7c8017e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:47.946207 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6d3f6-28c6-454b-b5b8-4a3fe7c8017e admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:47.946507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6d3f6-28c6-454b-b5b8-4a3fe7c8017e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:47.946792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6d3f6-28c6-454b-b5b8-4a3fe7c8017e admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:47.960180 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6d3f6-28c6-454b-b5b8-4a3fe7c8017e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:47.965083 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6d3f6-28c6-454b-b5b8-4a3fe7c8017e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:47.968132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 884/1788] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:52:47 2022] DELETE /identity/v3/users/3808aa2b8a674837b46505da8a3b1c6c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:48.007569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404ea77-d49c-416d-9faf-292ca2361d0b admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:48.011886 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-0404ea77-d49c-416d-9faf-292ca2361d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:48.015896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 905/1789] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:47 2022] POST /identity/v3/users => generated 288 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:48.165342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404ea77-d49c-416d-9faf-292ca2361d0b admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:48.170952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c166d5b-bf0a-44df-949c-81c509c6323c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:48.171736 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c166d5b-bf0a-44df-949c-81c509c6323c admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:48.172085 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c166d5b-bf0a-44df-949c-81c509c6323c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:48.172368 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c166d5b-bf0a-44df-949c-81c509c6323c admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:48.173118 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c166d5b-bf0a-44df-949c-81c509c6323c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:48.174916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c166d5b-bf0a-44df-949c-81c509c6323c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:48.193091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6d3f6-28c6-454b-b5b8-4a3fe7c8017e admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:48.194469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 885/1790] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:52:48.209067 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6d3f6-28c6-454b-b5b8-4a3fe7c8017e admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:48.216354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37cbc3a-90d5-4be4-b8e4-ef1da0fa5d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:48.217435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37cbc3a-90d5-4be4-b8e4-ef1da0fa5d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:48.217782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37cbc3a-90d5-4be4-b8e4-ef1da0fa5d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:48.218071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37cbc3a-90d5-4be4-b8e4-ef1da0fa5d63 admin admin] PATH_INFO: `/v3/projects/b9baed90990242feb5d3c5e5711d1d71/users/77f76282313b4a069ac7f821b0b86a88/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:48.251365 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37cbc3a-90d5-4be4-b8e4-ef1da0fa5d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9baed90990242feb5d3c5e5711d1d71, user_id=77f76282313b4a069ac7f821b0b86a88, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:48.256155 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37cbc3a-90d5-4be4-b8e4-ef1da0fa5d63 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:48.301895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c166d5b-bf0a-44df-949c-81c509c6323c admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:48.302086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 906/1791] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:48 2022] PUT /identity/v3/projects/b9baed90990242feb5d3c5e5711d1d71/users/77f76282313b4a069ac7f821b0b86a88/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:52:48.314271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c166d5b-bf0a-44df-949c-81c509c6323c admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:48.582451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a72884-ccbd-4d17-8e7d-29a64469410f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:48.583246 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a72884-ccbd-4d17-8e7d-29a64469410f admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:48.583523 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a72884-ccbd-4d17-8e7d-29a64469410f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:48.583878 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a72884-ccbd-4d17-8e7d-29a64469410f admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:48.584689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a72884-ccbd-4d17-8e7d-29a64469410f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:48.586406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a72884-ccbd-4d17-8e7d-29a64469410f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:48.644162 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37cbc3a-90d5-4be4-b8e4-ef1da0fa5d63 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:48.653484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 886/1792] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:48.661247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37cbc3a-90d5-4be4-b8e4-ef1da0fa5d63 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:48.678904 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35846ba-66ed-4b88-93f1-ea426ff6f2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:48.680254 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35846ba-66ed-4b88-93f1-ea426ff6f2ba admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:48.680753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35846ba-66ed-4b88-93f1-ea426ff6f2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:48.681409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35846ba-66ed-4b88-93f1-ea426ff6f2ba admin admin] PATH_INFO: `/v3/projects/b9baed90990242feb5d3c5e5711d1d71/users/77f76282313b4a069ac7f821b0b86a88/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:48.687934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35846ba-66ed-4b88-93f1-ea426ff6f2ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9baed90990242feb5d3c5e5711d1d71, user_id=77f76282313b4a069ac7f821b0b86a88, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:48.695502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35846ba-66ed-4b88-93f1-ea426ff6f2ba admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:48.743665 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc046a0f-0ccf-4fd2-a1a2-9e4f923cae94 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:48.745377 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc046a0f-0ccf-4fd2-a1a2-9e4f923cae94 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:48.745910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc046a0f-0ccf-4fd2-a1a2-9e4f923cae94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:48.747403 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 907/1793] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:48 2022] PUT /identity/v3/projects/b9baed90990242feb5d3c5e5711d1d71/users/77f76282313b4a069ac7f821b0b86a88/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:48.776163 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35846ba-66ed-4b88-93f1-ea426ff6f2ba admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:48.780154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e35846ba-66ed-4b88-93f1-ea426ff6f2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:48.820664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:48.829470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-bc046a0f-0ccf-4fd2-a1a2-9e4f923cae94 None None] MFA Rules not processed for user `77f76282313b4a069ac7f821b0b86a88`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:48.830752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-bc046a0f-0ccf-4fd2-a1a2-9e4f923cae94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:48.879178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:49.036720 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35846ba-66ed-4b88-93f1-ea426ff6f2ba admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:49.073208 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953b7cd-06ba-4956-9688-98201fae8df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:49.081087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953b7cd-06ba-4956-9688-98201fae8df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:49.081793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953b7cd-06ba-4956-9688-98201fae8df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:49.086040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953b7cd-06ba-4956-9688-98201fae8df4 admin admin] PATH_INFO: `/v3/projects/8e1c8c4d767648c18d469117390435c0` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:49.094042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953b7cd-06ba-4956-9688-98201fae8df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e1c8c4d767648c18d469117390435c0)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:49.103275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953b7cd-06ba-4956-9688-98201fae8df4 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:49.161154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 887/1794] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:48 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:49.268637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8953b7cd-06ba-4956-9688-98201fae8df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e1c8c4d767648c18d469117390435c0', 'request_id': 'req-8953b7cd-06ba-4956-9688-98201fae8df4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:49.296851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8953b7cd-06ba-4956-9688-98201fae8df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e1c8c4d767648c18d469117390435c0', 'request_id': 'req-8953b7cd-06ba-4956-9688-98201fae8df4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:49.316715 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d391c-0add-4e66-bf67-a3b65232eed3 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:49.317409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d391c-0add-4e66-bf67-a3b65232eed3 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:49.317409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d391c-0add-4e66-bf67-a3b65232eed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:49.326362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8953b7cd-06ba-4956-9688-98201fae8df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e1c8c4d767648c18d469117390435c0', 'request_id': 'req-8953b7cd-06ba-4956-9688-98201fae8df4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:49.368343 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8953b7cd-06ba-4956-9688-98201fae8df4 admin admin] The token cache is being invalidate because project 8e1c8c4d767648c18d469117390435c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:52:49.368644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8953b7cd-06ba-4956-9688-98201fae8df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:49.370569 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8953b7cd-06ba-4956-9688-98201fae8df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:49.380064 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 908/1795] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:52:48 2022] DELETE /identity/v3/projects/8e1c8c4d767648c18d469117390435c0 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:49.381589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-ae0d391c-0add-4e66-bf67-a3b65232eed3 None None] MFA Rules not processed for user `3a0d900f08d54098870ebbfa281e2a65`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:49.386235 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617aea4-730f-4c72-97d6-07b2b8c1877b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:49.386689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617aea4-730f-4c72-97d6-07b2b8c1877b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:49.387239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617aea4-730f-4c72-97d6-07b2b8c1877b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:49.398382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ae0d391c-0add-4e66-bf67-a3b65232eed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:49.463625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-8617aea4-730f-4c72-97d6-07b2b8c1877b None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:49.465043 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8617aea4-730f-4c72-97d6-07b2b8c1877b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:49.626045 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 888/1796] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:52:49.628296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d391c-0add-4e66-bf67-a3b65232eed3 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:49.630284 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ae0d391c-0add-4e66-bf67-a3b65232eed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:49.741004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 909/1797] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:49.751195 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc98c09-ae69-452c-b093-9e0890958528 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:49.751490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc98c09-ae69-452c-b093-9e0890958528 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:49.762535 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc98c09-ae69-452c-b093-9e0890958528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:49.865938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-3fc98c09-ae69-452c-b093-9e0890958528 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:49.867678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-3fc98c09-ae69-452c-b093-9e0890958528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:49.891612 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 910/1798] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:49.896237 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc98c09-ae69-452c-b093-9e0890958528 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:49.925324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d391c-0add-4e66-bf67-a3b65232eed3 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:49.934859 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8112b-a076-4164-a7bb-44ded1fe7bd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:49.938036 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8112b-a076-4164-a7bb-44ded1fe7bd3 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:49.938831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8112b-a076-4164-a7bb-44ded1fe7bd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:49.939126 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8112b-a076-4164-a7bb-44ded1fe7bd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:49.944968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-47b8112b-a076-4164-a7bb-44ded1fe7bd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:49.945742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc98c09-ae69-452c-b093-9e0890958528 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:49.964194 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174926e4-3a01-4fd6-a994-f14636de8ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:49.965170 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174926e4-3a01-4fd6-a994-f14636de8ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:49.965652 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174926e4-3a01-4fd6-a994-f14636de8ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:49.966137 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174926e4-3a01-4fd6-a994-f14636de8ce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:49.967040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174926e4-3a01-4fd6-a994-f14636de8ce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:52:49.967501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174926e4-3a01-4fd6-a994-f14636de8ce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:49.969585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174926e4-3a01-4fd6-a994-f14636de8ce8 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:49.984393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 911/1799] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:52:49.986669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730886bb-b7f6-4df6-b345-3718d27cbf06 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:49.987108 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730886bb-b7f6-4df6-b345-3718d27cbf06 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:49.987618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730886bb-b7f6-4df6-b345-3718d27cbf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:50.079432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:50.155284 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-730886bb-b7f6-4df6-b345-3718d27cbf06 None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:50.156684 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-730886bb-b7f6-4df6-b345-3718d27cbf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:50.159934 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8112b-a076-4164-a7bb-44ded1fe7bd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:50.162573 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8112b-a076-4164-a7bb-44ded1fe7bd3 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:50.206053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 889/1800] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:49 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:50.207351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8112b-a076-4164-a7bb-44ded1fe7bd3 service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:50.258561 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8112b-a076-4164-a7bb-44ded1fe7bd3 service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:50.278185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43083a13-8ee5-4f9e-bf14-b88fca14f397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:50.280118 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43083a13-8ee5-4f9e-bf14-b88fca14f397 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:50.280501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43083a13-8ee5-4f9e-bf14-b88fca14f397 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:50.281321 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43083a13-8ee5-4f9e-bf14-b88fca14f397 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:50.283896 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43083a13-8ee5-4f9e-bf14-b88fca14f397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:50.285953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43083a13-8ee5-4f9e-bf14-b88fca14f397 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:50.299573 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:50.393622 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 912/1801] 10.222.0.60 () {58 vars in 1009 bytes} [Thu Jul 21 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:50.395323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730886bb-b7f6-4df6-b345-3718d27cbf06 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:50.397302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 890/1802] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:50 2022] POST /identity/v3/projects => generated 375 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:50.404110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43083a13-8ee5-4f9e-bf14-b88fca14f397 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:50.423598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730886bb-b7f6-4df6-b345-3718d27cbf06 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:50.594909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5b3e1-bdc9-42b3-957b-291461c574ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:50.595510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5b3e1-bdc9-42b3-957b-291461c574ba service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:50.595819 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5b3e1-bdc9-42b3-957b-291461c574ba service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:50.596434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5b3e1-bdc9-42b3-957b-291461c574ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:50.620981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-0ce5b3e1-bdc9-42b3-957b-291461c574ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:50.742170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43083a13-8ee5-4f9e-bf14-b88fca14f397 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:50.774763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a51000-3410-4a73-9ae7-f0956c034dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:50.777414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a51000-3410-4a73-9ae7-f0956c034dbb admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:50.777884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a51000-3410-4a73-9ae7-f0956c034dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:50.778377 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a51000-3410-4a73-9ae7-f0956c034dbb admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:50.788070 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a51000-3410-4a73-9ae7-f0956c034dbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:50.788346 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a51000-3410-4a73-9ae7-f0956c034dbb admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:50.805046 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5b3e1-bdc9-42b3-957b-291461c574ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:50.811254 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5b3e1-bdc9-42b3-957b-291461c574ba service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:50.834244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 913/1803] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:50.834898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5b3e1-bdc9-42b3-957b-291461c574ba service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:50.840816 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-0ce5b3e1-bdc9-42b3-957b-291461c574ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:50.861328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 891/1804] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:50 2022] POST /identity/v3/users => generated 288 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:50.863309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a51000-3410-4a73-9ae7-f0956c034dbb admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:50.897013 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a51000-3410-4a73-9ae7-f0956c034dbb admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:50.920193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058ad33-6485-4a3f-bf8f-e2e07b514f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:50.921479 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058ad33-6485-4a3f-bf8f-e2e07b514f6a admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:50.923196 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058ad33-6485-4a3f-bf8f-e2e07b514f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:50.924001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058ad33-6485-4a3f-bf8f-e2e07b514f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:50.925914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058ad33-6485-4a3f-bf8f-e2e07b514f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:50.929834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058ad33-6485-4a3f-bf8f-e2e07b514f6a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:50.974683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 892/1805] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:50.982307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058ad33-6485-4a3f-bf8f-e2e07b514f6a admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:51.018867 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058ad33-6485-4a3f-bf8f-e2e07b514f6a admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:51.031267 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea946db6-67d7-4e59-9152-e82ce401c320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:51.033227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea946db6-67d7-4e59-9152-e82ce401c320 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:51.036731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea946db6-67d7-4e59-9152-e82ce401c320 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:51.037293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea946db6-67d7-4e59-9152-e82ce401c320 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:51.039738 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea946db6-67d7-4e59-9152-e82ce401c320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:51.043562 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea946db6-67d7-4e59-9152-e82ce401c320 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:51.058549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:51.113177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 893/1806] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:51.117044 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea946db6-67d7-4e59-9152-e82ce401c320 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:51.136870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea946db6-67d7-4e59-9152-e82ce401c320 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:51.159556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dbc98-1cce-4f29-b1a7-4342c3048b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:51.161719 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dbc98-1cce-4f29-b1a7-4342c3048b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:51.162908 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dbc98-1cce-4f29-b1a7-4342c3048b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:51.163937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dbc98-1cce-4f29-b1a7-4342c3048b79 admin admin] PATH_INFO: `/v3/projects/f1a1374137d844b5abb2db2cad1f1851/users/95800b197dac4da287e86fe2e9e19fb2/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:51.173094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5b3e1-bdc9-42b3-957b-291461c574ba service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:51.179371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379c878-6208-45d0-9e1b-8442d5ce19d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:51.180085 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379c878-6208-45d0-9e1b-8442d5ce19d8 service swift] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:51.180375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379c878-6208-45d0-9e1b-8442d5ce19d8 service swift] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:51.180655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379c878-6208-45d0-9e1b-8442d5ce19d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:51.231354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12dbc98-1cce-4f29-b1a7-4342c3048b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a1374137d844b5abb2db2cad1f1851, user_id=95800b197dac4da287e86fe2e9e19fb2, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:51.234542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12dbc98-1cce-4f29-b1a7-4342c3048b79 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:51.284304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 894/1807] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:51 2022] PUT /identity/v3/projects/f1a1374137d844b5abb2db2cad1f1851/users/95800b197dac4da287e86fe2e9e19fb2/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:51.286852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3d515-ef54-4318-8100-65365a75e79e None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:51.287151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3d515-ef54-4318-8100-65365a75e79e None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:51.287434 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3d515-ef54-4318-8100-65365a75e79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:51.341312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379c878-6208-45d0-9e1b-8442d5ce19d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:51.345899 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379c878-6208-45d0-9e1b-8442d5ce19d8 service swift] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:51.373157 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d3d3d515-ef54-4318-8100-65365a75e79e None None] MFA Rules not processed for user `95800b197dac4da287e86fe2e9e19fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:51.384117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d3d3d515-ef54-4318-8100-65365a75e79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:51.407254 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 914/1808] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 20:52:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:52:51.646752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 895/1809] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:51.651572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379c878-6208-45d0-9e1b-8442d5ce19d8 service swift] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:51.679587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379c878-6208-45d0-9e1b-8442d5ce19d8 service swift] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:51.860544 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974030e-b5f5-41db-9df7-b24b80d6764f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:51.861551 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974030e-b5f5-41db-9df7-b24b80d6764f admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:51.861763 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974030e-b5f5-41db-9df7-b24b80d6764f admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:51.861985 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974030e-b5f5-41db-9df7-b24b80d6764f admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:51.862699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974030e-b5f5-41db-9df7-b24b80d6764f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:51.864837 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974030e-b5f5-41db-9df7-b24b80d6764f admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:51.940108 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 915/1810] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:52:51 2022] POST /identity/v3/projects => generated 377 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:51.946884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3d515-ef54-4318-8100-65365a75e79e None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:51.982052 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3d515-ef54-4318-8100-65365a75e79e None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:52.059373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:52.227236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8484865-967f-4d32-b953-6110ca239a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:52.228006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8484865-967f-4d32-b953-6110ca239a6b admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:52.229436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8484865-967f-4d32-b953-6110ca239a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:52.229761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8484865-967f-4d32-b953-6110ca239a6b admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:52.240929 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8484865-967f-4d32-b953-6110ca239a6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:52.242696 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8484865-967f-4d32-b953-6110ca239a6b admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:52.310104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 896/1811] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:52:51 2022] POST /identity/v3/users => generated 295 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:52:52.311957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974030e-b5f5-41db-9df7-b24b80d6764f admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:52.346563 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974030e-b5f5-41db-9df7-b24b80d6764f admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:52.376963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814334a-3010-41f3-8422-a810e701cd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:52.386600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814334a-3010-41f3-8422-a810e701cd05 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:52.388709 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814334a-3010-41f3-8422-a810e701cd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:52.391653 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814334a-3010-41f3-8422-a810e701cd05 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:52.404246 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814334a-3010-41f3-8422-a810e701cd05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:52.404246 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814334a-3010-41f3-8422-a810e701cd05 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:52.463998 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 916/1812] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:52.464717 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8484865-967f-4d32-b953-6110ca239a6b admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:52.485699 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8484865-967f-4d32-b953-6110ca239a6b admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:52.492492 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eabdb7-b276-469b-ae43-269f8b97d9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:52.494133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eabdb7-b276-469b-ae43-269f8b97d9d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:52.494433 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eabdb7-b276-469b-ae43-269f8b97d9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:52.494714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eabdb7-b276-469b-ae43-269f8b97d9d2 admin admin] PATH_INFO: `/v3/projects/e7b5dae3a3bd4093bb198c39e4960af1/users/802d418d83534e7aa5ab9f10a7309663/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:52.522914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eabdb7-b276-469b-ae43-269f8b97d9d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7b5dae3a3bd4093bb198c39e4960af1, user_id=802d418d83534e7aa5ab9f10a7309663, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:52.526801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eabdb7-b276-469b-ae43-269f8b97d9d2 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:52.569388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 897/1813] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:52 2022] PUT /identity/v3/projects/e7b5dae3a3bd4093bb198c39e4960af1/users/802d418d83534e7aa5ab9f10a7309663/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:52:52.577772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814334a-3010-41f3-8422-a810e701cd05 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:52.631301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814334a-3010-41f3-8422-a810e701cd05 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:52.877723 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:52.918553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419621f6-701c-4eda-a3f8-af476890b47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:52.919664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419621f6-701c-4eda-a3f8-af476890b47a admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:52.920189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419621f6-701c-4eda-a3f8-af476890b47a admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:52.920683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419621f6-701c-4eda-a3f8-af476890b47a admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:52.921816 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419621f6-701c-4eda-a3f8-af476890b47a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:52.935506 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419621f6-701c-4eda-a3f8-af476890b47a admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:52.988532 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 917/1814] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:52:52.988716 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eabdb7-b276-469b-ae43-269f8b97d9d2 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:53.019129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eabdb7-b276-469b-ae43-269f8b97d9d2 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:53.037630 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79c807-54a2-48eb-9596-377ad6f09c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:53.039179 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79c807-54a2-48eb-9596-377ad6f09c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:53.039464 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79c807-54a2-48eb-9596-377ad6f09c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:53.039837 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79c807-54a2-48eb-9596-377ad6f09c1b admin admin] PATH_INFO: `/v3/projects/e7b5dae3a3bd4093bb198c39e4960af1/users/802d418d83534e7aa5ab9f10a7309663/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:53.048714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79c807-54a2-48eb-9596-377ad6f09c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7b5dae3a3bd4093bb198c39e4960af1, user_id=802d418d83534e7aa5ab9f10a7309663, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:53.055786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79c807-54a2-48eb-9596-377ad6f09c1b admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:53.149340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 898/1815] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:52:52 2022] PUT /identity/v3/projects/e7b5dae3a3bd4093bb198c39e4960af1/users/802d418d83534e7aa5ab9f10a7309663/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:53.161149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e488a10-40f3-4a80-9490-f84988550322 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:53.163467 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e488a10-40f3-4a80-9490-f84988550322 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:53.164197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e488a10-40f3-4a80-9490-f84988550322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:53.227938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-7e488a10-40f3-4a80-9490-f84988550322 None None] MFA Rules not processed for user `802d418d83534e7aa5ab9f10a7309663`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:53.230033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-7e488a10-40f3-4a80-9490-f84988550322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:53.498181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 918/1816] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:53.530149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79c807-54a2-48eb-9596-377ad6f09c1b admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:53.746463 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79c807-54a2-48eb-9596-377ad6f09c1b admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:53.776150 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f4a44-d155-4501-b9af-94279261a491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:53.777191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f4a44-d155-4501-b9af-94279261a491 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:53.777690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f4a44-d155-4501-b9af-94279261a491 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:53.780262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f4a44-d155-4501-b9af-94279261a491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:53.961430 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f4a44-d155-4501-b9af-94279261a491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:53.966019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f4a44-d155-4501-b9af-94279261a491 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:53.988745 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 899/1817] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:57.754717 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e488a10-40f3-4a80-9490-f84988550322 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:57.758055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-7e488a10-40f3-4a80-9490-f84988550322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:57.852492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e488a10-40f3-4a80-9490-f84988550322 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:57.859407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b56737-96ae-448d-a8af-f6e8dbc3797d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:57.860369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b56737-96ae-448d-a8af-f6e8dbc3797d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:57.860664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b56737-96ae-448d-a8af-f6e8dbc3797d admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:57.861188 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b56737-96ae-448d-a8af-f6e8dbc3797d admin admin] PATH_INFO: `/v3/users/bc6344e800394e0c8d19318b76d7d650` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:57.862473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b56737-96ae-448d-a8af-f6e8dbc3797d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc6344e800394e0c8d19318b76d7d650)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:57.871646 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b56737-96ae-448d-a8af-f6e8dbc3797d admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:58.004026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-10b56737-96ae-448d-a8af-f6e8dbc3797d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc6344e800394e0c8d19318b76d7d650'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:58.005417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-10b56737-96ae-448d-a8af-f6e8dbc3797d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc6344e800394e0c8d19318b76d7d650'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:58.053362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-10b56737-96ae-448d-a8af-f6e8dbc3797d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc6344e800394e0c8d19318b76d7d650'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:58.065149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:58.069417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-10b56737-96ae-448d-a8af-f6e8dbc3797d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc6344e800394e0c8d19318b76d7d650'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:58.080715 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-10b56737-96ae-448d-a8af-f6e8dbc3797d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc6344e800394e0c8d19318b76d7d650'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:58.090601 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 919/1818] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:52:57 2022] DELETE /identity/v3/users/bc6344e800394e0c8d19318b76d7d650 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:58.095383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e5e4e-af78-450d-9ee3-37e0c56022ea None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:58.096049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e5e4e-af78-450d-9ee3-37e0c56022ea None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:58.096049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e5e4e-af78-450d-9ee3-37e0c56022ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:58.117713 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-6d0e5e4e-af78-450d-9ee3-37e0c56022ea None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:58.119666 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6d0e5e4e-af78-450d-9ee3-37e0c56022ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:58.140387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:58.187012 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 900/1819] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:58.196476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b56737-96ae-448d-a8af-f6e8dbc3797d admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:58.198901 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-10b56737-96ae-448d-a8af-f6e8dbc3797d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:58.267855 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b56737-96ae-448d-a8af-f6e8dbc3797d admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:58.274369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25148f50-4517-40ef-b3c3-ca6505201363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:58.276869 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25148f50-4517-40ef-b3c3-ca6505201363 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:58.277321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25148f50-4517-40ef-b3c3-ca6505201363 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:58.280081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25148f50-4517-40ef-b3c3-ca6505201363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:58.292973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25148f50-4517-40ef-b3c3-ca6505201363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:58.294420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25148f50-4517-40ef-b3c3-ca6505201363 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:58.304608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 920/1820] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:52:58.575210 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e5e4e-af78-450d-9ee3-37e0c56022ea None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:58.577292 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6d0e5e4e-af78-450d-9ee3-37e0c56022ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:58.592081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e5e4e-af78-450d-9ee3-37e0c56022ea None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:58.599748 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90070042-2f7c-4642-8daa-aca8a55bbca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:58.600723 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90070042-2f7c-4642-8daa-aca8a55bbca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:58.601174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90070042-2f7c-4642-8daa-aca8a55bbca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:58.601639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90070042-2f7c-4642-8daa-aca8a55bbca1 admin admin] PATH_INFO: `/v3/users/b9fff2fdf78b4e35b20980e577e1af9f` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:58.602912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90070042-2f7c-4642-8daa-aca8a55bbca1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9fff2fdf78b4e35b20980e577e1af9f)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:58.604913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90070042-2f7c-4642-8daa-aca8a55bbca1 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:58.692798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-90070042-2f7c-4642-8daa-aca8a55bbca1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9fff2fdf78b4e35b20980e577e1af9f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:58.693807 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25148f50-4517-40ef-b3c3-ca6505201363 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:58.695359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-25148f50-4517-40ef-b3c3-ca6505201363 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:58.711904 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25148f50-4517-40ef-b3c3-ca6505201363 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:58.715466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-90070042-2f7c-4642-8daa-aca8a55bbca1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9fff2fdf78b4e35b20980e577e1af9f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:58.716080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-90070042-2f7c-4642-8daa-aca8a55bbca1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9fff2fdf78b4e35b20980e577e1af9f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:58.723791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be067f-245f-4973-aba3-229537190881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:58.725052 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be067f-245f-4973-aba3-229537190881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:58.725420 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-90070042-2f7c-4642-8daa-aca8a55bbca1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9fff2fdf78b4e35b20980e577e1af9f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:58.725697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be067f-245f-4973-aba3-229537190881 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:58.726052 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be067f-245f-4973-aba3-229537190881 admin admin] PATH_INFO: `/v3/projects/a84fe5a2425c43f581bf505c96f43ee3` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:58.726307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-90070042-2f7c-4642-8daa-aca8a55bbca1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9fff2fdf78b4e35b20980e577e1af9f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:58.727947 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be067f-245f-4973-aba3-229537190881 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a84fe5a2425c43f581bf505c96f43ee3)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:58.730900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be067f-245f-4973-aba3-229537190881 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:58.784038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 901/1821] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:52:58 2022] DELETE /identity/v3/users/b9fff2fdf78b4e35b20980e577e1af9f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:58.785920 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af400a66-8a44-4d4d-88ff-8a69059e3073 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:58.786287 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af400a66-8a44-4d4d-88ff-8a69059e3073 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:58.786585 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af400a66-8a44-4d4d-88ff-8a69059e3073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:58.810373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-08be067f-245f-4973-aba3-229537190881 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a84fe5a2425c43f581bf505c96f43ee3', 'request_id': 'req-08be067f-245f-4973-aba3-229537190881'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:58.827470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-08be067f-245f-4973-aba3-229537190881 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a84fe5a2425c43f581bf505c96f43ee3', 'request_id': 'req-08be067f-245f-4973-aba3-229537190881'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:58.833897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-af400a66-8a44-4d4d-88ff-8a69059e3073 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:58.835650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-08be067f-245f-4973-aba3-229537190881 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a84fe5a2425c43f581bf505c96f43ee3', 'request_id': 'req-08be067f-245f-4973-aba3-229537190881'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:58.836153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-af400a66-8a44-4d4d-88ff-8a69059e3073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:58.856482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-08be067f-245f-4973-aba3-229537190881 admin admin] The token cache is being invalidate because project a84fe5a2425c43f581bf505c96f43ee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:52:58.856880 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-08be067f-245f-4973-aba3-229537190881 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:58.857645 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-08be067f-245f-4973-aba3-229537190881 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:58.859506 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 921/1822] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:52:58 2022] DELETE /identity/v3/projects/a84fe5a2425c43f581bf505c96f43ee3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:52:58.863694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be067f-245f-4973-aba3-229537190881 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:58.866363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-08be067f-245f-4973-aba3-229537190881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:58.932423 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 902/1823] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:52:58.940369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af400a66-8a44-4d4d-88ff-8a69059e3073 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:58.942630 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-af400a66-8a44-4d4d-88ff-8a69059e3073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:58.988444 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be067f-245f-4973-aba3-229537190881 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:59.000944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d5cac-e8b3-430d-992a-a39e644ff383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:59.001960 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d5cac-e8b3-430d-992a-a39e644ff383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:59.002418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d5cac-e8b3-430d-992a-a39e644ff383 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:59.002712 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d5cac-e8b3-430d-992a-a39e644ff383 admin admin] PATH_INFO: `/v3/users/77f76282313b4a069ac7f821b0b86a88` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:59.008560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d5cac-e8b3-430d-992a-a39e644ff383 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77f76282313b4a069ac7f821b0b86a88)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:59.011114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d5cac-e8b3-430d-992a-a39e644ff383 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:59.015392 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:59.030606 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:52:59.048680 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af400a66-8a44-4d4d-88ff-8a69059e3073 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:59.054071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61900dff-11af-4845-a49d-df4b0dd9d0f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:59.054712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61900dff-11af-4845-a49d-df4b0dd9d0f5 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:59.054994 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61900dff-11af-4845-a49d-df4b0dd9d0f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:59.055272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61900dff-11af-4845-a49d-df4b0dd9d0f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:59.066082 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61900dff-11af-4845-a49d-df4b0dd9d0f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:59.067765 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61900dff-11af-4845-a49d-df4b0dd9d0f5 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:59.086889 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 903/1824] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:52:59.096491 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e18d5cac-e8b3-430d-992a-a39e644ff383 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77f76282313b4a069ac7f821b0b86a88'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:59.098083 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e18d5cac-e8b3-430d-992a-a39e644ff383 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77f76282313b4a069ac7f821b0b86a88'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:59.125239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e18d5cac-e8b3-430d-992a-a39e644ff383 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77f76282313b4a069ac7f821b0b86a88'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:59.136584 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e18d5cac-e8b3-430d-992a-a39e644ff383 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77f76282313b4a069ac7f821b0b86a88'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:59.147107 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e18d5cac-e8b3-430d-992a-a39e644ff383 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77f76282313b4a069ac7f821b0b86a88'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:59.161707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 922/1825] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:52:58 2022] DELETE /identity/v3/users/77f76282313b4a069ac7f821b0b86a88 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:59.586697 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61900dff-11af-4845-a49d-df4b0dd9d0f5 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:59.589639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-61900dff-11af-4845-a49d-df4b0dd9d0f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:59.614777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61900dff-11af-4845-a49d-df4b0dd9d0f5 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:59.722238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d1817-cc05-4508-bf89-4a7275c602e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:59.723539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d1817-cc05-4508-bf89-4a7275c602e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:59.723927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d1817-cc05-4508-bf89-4a7275c602e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:59.724098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d1817-cc05-4508-bf89-4a7275c602e0 admin admin] PATH_INFO: `/v3/projects/bf7b8f3c6f4b4223b52dabcde2cc73b5` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:59.726234 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d1817-cc05-4508-bf89-4a7275c602e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf7b8f3c6f4b4223b52dabcde2cc73b5)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:59.728545 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d1817-cc05-4508-bf89-4a7275c602e0 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:59.734233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d5cac-e8b3-430d-992a-a39e644ff383 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:59.736180 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e18d5cac-e8b3-430d-992a-a39e644ff383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:59.759564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d5cac-e8b3-430d-992a-a39e644ff383 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:59.769577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57984e-f71e-4b42-8579-e228c42b4a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:52:59.770894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57984e-f71e-4b42-8579-e228c42b4a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:59.771413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57984e-f71e-4b42-8579-e228c42b4a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:59.772653 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57984e-f71e-4b42-8579-e228c42b4a42 admin admin] PATH_INFO: `/v3/projects/b9baed90990242feb5d3c5e5711d1d71` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:59.774499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57984e-f71e-4b42-8579-e228c42b4a42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9baed90990242feb5d3c5e5711d1d71)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:52:59.778802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57984e-f71e-4b42-8579-e228c42b4a42 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:52:59.808551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-939d1817-cc05-4508-bf89-4a7275c602e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf7b8f3c6f4b4223b52dabcde2cc73b5', 'request_id': 'req-939d1817-cc05-4508-bf89-4a7275c602e0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:59.817896 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-939d1817-cc05-4508-bf89-4a7275c602e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf7b8f3c6f4b4223b52dabcde2cc73b5', 'request_id': 'req-939d1817-cc05-4508-bf89-4a7275c602e0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:59.836764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-939d1817-cc05-4508-bf89-4a7275c602e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf7b8f3c6f4b4223b52dabcde2cc73b5', 'request_id': 'req-939d1817-cc05-4508-bf89-4a7275c602e0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:59.843414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9c57984e-f71e-4b42-8579-e228c42b4a42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9baed90990242feb5d3c5e5711d1d71', 'request_id': 'req-9c57984e-f71e-4b42-8579-e228c42b4a42'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:59.851624 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9c57984e-f71e-4b42-8579-e228c42b4a42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9baed90990242feb5d3c5e5711d1d71', 'request_id': 'req-9c57984e-f71e-4b42-8579-e228c42b4a42'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:59.853144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-939d1817-cc05-4508-bf89-4a7275c602e0 admin admin] The token cache is being invalidate because project bf7b8f3c6f4b4223b52dabcde2cc73b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:52:59.853753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-939d1817-cc05-4508-bf89-4a7275c602e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:59.854894 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-939d1817-cc05-4508-bf89-4a7275c602e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:59.856725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 904/1826] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:52:59 2022] DELETE /identity/v3/projects/bf7b8f3c6f4b4223b52dabcde2cc73b5 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:59.859191 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9c57984e-f71e-4b42-8579-e228c42b4a42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9baed90990242feb5d3c5e5711d1d71', 'request_id': 'req-9c57984e-f71e-4b42-8579-e228c42b4a42'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:59.860494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d1817-cc05-4508-bf89-4a7275c602e0 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:52:59.863270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-939d1817-cc05-4508-bf89-4a7275c602e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:59.877417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9c57984e-f71e-4b42-8579-e228c42b4a42 admin admin] The token cache is being invalidate because project b9baed90990242feb5d3c5e5711d1d71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:52:59.877932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9c57984e-f71e-4b42-8579-e228c42b4a42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:59.878679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9c57984e-f71e-4b42-8579-e228c42b4a42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:52:59.880384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 923/1827] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:52:59 2022] DELETE /identity/v3/projects/b9baed90990242feb5d3c5e5711d1d71 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:52:59.886308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91291766-25a1-4d47-998e-72cedff3ebc9 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:52:59.886666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91291766-25a1-4d47-998e-72cedff3ebc9 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:52:59.887029 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91291766-25a1-4d47-998e-72cedff3ebc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:52:59.909358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-91291766-25a1-4d47-998e-72cedff3ebc9 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:52:59.910615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-91291766-25a1-4d47-998e-72cedff3ebc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:52:59.956591 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d1817-cc05-4508-bf89-4a7275c602e0 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:52:59.973061 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:00.017998 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 924/1828] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:00.028338 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2c9e5-48a7-40cd-8f7f-dd795cf94a9b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.028991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2c9e5-48a7-40cd-8f7f-dd795cf94a9b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.029617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2c9e5-48a7-40cd-8f7f-dd795cf94a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.036793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb00ec-58db-4340-b9d7-22e40336f910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:00.038028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb00ec-58db-4340-b9d7-22e40336f910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.039182 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb00ec-58db-4340-b9d7-22e40336f910 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.039580 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb00ec-58db-4340-b9d7-22e40336f910 admin admin] PATH_INFO: `/v3/users/6cd72c82e19942d2811996192bb8a2a9` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.041103 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb00ec-58db-4340-b9d7-22e40336f910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cd72c82e19942d2811996192bb8a2a9)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:00.043601 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb00ec-58db-4340-b9d7-22e40336f910 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:00.050669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-cdf2c9e5-48a7-40cd-8f7f-dd795cf94a9b None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:00.052419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-cdf2c9e5-48a7-40cd-8f7f-dd795cf94a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:00.063096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 925/1829] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:00.070180 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2c9e5-48a7-40cd-8f7f-dd795cf94a9b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:00.083258 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2c9e5-48a7-40cd-8f7f-dd795cf94a9b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:00.088381 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e824c0-2150-40c6-8497-846b8d2c4e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:00.089157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e824c0-2150-40c6-8497-846b8d2c4e7c admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.090233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e824c0-2150-40c6-8497-846b8d2c4e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.090233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e824c0-2150-40c6-8497-846b8d2c4e7c admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.092021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e824c0-2150-40c6-8497-846b8d2c4e7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:53:00.092334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e824c0-2150-40c6-8497-846b8d2c4e7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:00.094629 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e824c0-2150-40c6-8497-846b8d2c4e7c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:00.097980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:00.112089 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 926/1830] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:53:00.116582 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e824c0-2150-40c6-8497-846b8d2c4e7c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:00.123592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a4cb00ec-58db-4340-b9d7-22e40336f910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cd72c82e19942d2811996192bb8a2a9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:00.131946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e824c0-2150-40c6-8497-846b8d2c4e7c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:00.139426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a4cb00ec-58db-4340-b9d7-22e40336f910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cd72c82e19942d2811996192bb8a2a9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:00.140374 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a4cb00ec-58db-4340-b9d7-22e40336f910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cd72c82e19942d2811996192bb8a2a9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:00.150512 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c1ae2-3ff0-4de4-bce5-ca267cc41b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:00.151829 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c1ae2-3ff0-4de4-bce5-ca267cc41b3b admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.152642 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c1ae2-3ff0-4de4-bce5-ca267cc41b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.153045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c1ae2-3ff0-4de4-bce5-ca267cc41b3b admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.154583 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c1ae2-3ff0-4de4-bce5-ca267cc41b3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:00.158258 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c1ae2-3ff0-4de4-bce5-ca267cc41b3b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:00.158542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a4cb00ec-58db-4340-b9d7-22e40336f910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cd72c82e19942d2811996192bb8a2a9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:00.159826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a4cb00ec-58db-4340-b9d7-22e40336f910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cd72c82e19942d2811996192bb8a2a9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:00.199804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 905/1831] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:52:59 2022] DELETE /identity/v3/users/6cd72c82e19942d2811996192bb8a2a9 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:53:00.204286 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 927/1832] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:00 2022] POST /identity/v3/projects => generated 383 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:53:00.215437 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb00ec-58db-4340-b9d7-22e40336f910 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:00.216995 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a4cb00ec-58db-4340-b9d7-22e40336f910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:00.322530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb00ec-58db-4340-b9d7-22e40336f910 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:00.368495 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:00.386484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b5bf2-1157-4b02-8f0c-fadb1624e4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:00.387207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b5bf2-1157-4b02-8f0c-fadb1624e4c9 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.387494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b5bf2-1157-4b02-8f0c-fadb1624e4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.387804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b5bf2-1157-4b02-8f0c-fadb1624e4c9 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.388510 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b5bf2-1157-4b02-8f0c-fadb1624e4c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:00.390470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b5bf2-1157-4b02-8f0c-fadb1624e4c9 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:00.422806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 906/1833] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:00 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:00.424209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c1ae2-3ff0-4de4-bce5-ca267cc41b3b admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:00.437367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c1ae2-3ff0-4de4-bce5-ca267cc41b3b admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:00.443867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa719c-c5fb-4169-8ab0-6ae27183e575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:00.444792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa719c-c5fb-4169-8ab0-6ae27183e575 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.445290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa719c-c5fb-4169-8ab0-6ae27183e575 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.445623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa719c-c5fb-4169-8ab0-6ae27183e575 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.446657 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa719c-c5fb-4169-8ab0-6ae27183e575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:00.448698 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aa719c-c5fb-4169-8ab0-6ae27183e575 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:00.467759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 928/1834] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:00.473466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b5bf2-1157-4b02-8f0c-fadb1624e4c9 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:00.490308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b5bf2-1157-4b02-8f0c-fadb1624e4c9 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:00.496980 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8d92f-6cbd-4a6a-b05a-a997ae462c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:00.497657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8d92f-6cbd-4a6a-b05a-a997ae462c16 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.497937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8d92f-6cbd-4a6a-b05a-a997ae462c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.498209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8d92f-6cbd-4a6a-b05a-a997ae462c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.498951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8d92f-6cbd-4a6a-b05a-a997ae462c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:00.500585 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8d92f-6cbd-4a6a-b05a-a997ae462c16 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:00.511458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 907/1835] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:00.515933 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa719c-c5fb-4169-8ab0-6ae27183e575 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:00.528399 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa719c-c5fb-4169-8ab0-6ae27183e575 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:00.529313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8d92f-6cbd-4a6a-b05a-a997ae462c16 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:00.531008 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8bc8d92f-6cbd-4a6a-b05a-a997ae462c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:00.533495 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6133fe0-5322-47de-9d7f-90d93c742d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:00.534455 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6133fe0-5322-47de-9d7f-90d93c742d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.534738 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6133fe0-5322-47de-9d7f-90d93c742d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.535018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6133fe0-5322-47de-9d7f-90d93c742d35 admin admin] PATH_INFO: `/v3/projects/f570b02b4fac441eb4e7a5a7c38c9ba4/users/067061404a7a412a80cee7ded0f94263/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.546864 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8d92f-6cbd-4a6a-b05a-a997ae462c16 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:00.555275 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e81ed2-eab3-489b-ac11-18127b255cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:00.557677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6133fe0-5322-47de-9d7f-90d93c742d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f570b02b4fac441eb4e7a5a7c38c9ba4, user_id=067061404a7a412a80cee7ded0f94263, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:00.559873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e81ed2-eab3-489b-ac11-18127b255cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.560169 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e81ed2-eab3-489b-ac11-18127b255cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.560452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e81ed2-eab3-489b-ac11-18127b255cbb admin admin] PATH_INFO: `/v3/projects/54ce6ee876cd4376a8bc1cda110d6298` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.561952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e81ed2-eab3-489b-ac11-18127b255cbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=54ce6ee876cd4376a8bc1cda110d6298)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:00.562176 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6133fe0-5322-47de-9d7f-90d93c742d35 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:00.564169 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e81ed2-eab3-489b-ac11-18127b255cbb admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:00.579145 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 929/1836] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:00 2022] PUT /identity/v3/projects/f570b02b4fac441eb4e7a5a7c38c9ba4/users/067061404a7a412a80cee7ded0f94263/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:53:00.590181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81665d-0d1b-489b-856f-c9b90a5de56b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.590469 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81665d-0d1b-489b-856f-c9b90a5de56b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.590745 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81665d-0d1b-489b-856f-c9b90a5de56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.619983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-da81665d-0d1b-489b-856f-c9b90a5de56b None None] MFA Rules not processed for user `067061404a7a412a80cee7ded0f94263`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:00.621140 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-da81665d-0d1b-489b-856f-c9b90a5de56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:00.622449 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-89e81ed2-eab3-489b-ac11-18127b255cbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54ce6ee876cd4376a8bc1cda110d6298', 'request_id': 'req-89e81ed2-eab3-489b-ac11-18127b255cbb'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:00.631184 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-89e81ed2-eab3-489b-ac11-18127b255cbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54ce6ee876cd4376a8bc1cda110d6298', 'request_id': 'req-89e81ed2-eab3-489b-ac11-18127b255cbb'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:00.638429 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-89e81ed2-eab3-489b-ac11-18127b255cbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54ce6ee876cd4376a8bc1cda110d6298', 'request_id': 'req-89e81ed2-eab3-489b-ac11-18127b255cbb'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:00.650769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-89e81ed2-eab3-489b-ac11-18127b255cbb admin admin] The token cache is being invalidate because project 54ce6ee876cd4376a8bc1cda110d6298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:00.651054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-89e81ed2-eab3-489b-ac11-18127b255cbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:00.651762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-89e81ed2-eab3-489b-ac11-18127b255cbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:00.653436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 908/1837] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:00 2022] DELETE /identity/v3/projects/54ce6ee876cd4376a8bc1cda110d6298 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:53:00.658572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86284bc0-61ba-43b6-b147-43a99cbab0a7 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.659163 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86284bc0-61ba-43b6-b147-43a99cbab0a7 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.659681 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86284bc0-61ba-43b6-b147-43a99cbab0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.678062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-86284bc0-61ba-43b6-b147-43a99cbab0a7 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:00.679317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-86284bc0-61ba-43b6-b147-43a99cbab0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:00.689214 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 930/1838] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:00.693677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81665d-0d1b-489b-856f-c9b90a5de56b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:00.694980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-da81665d-0d1b-489b-856f-c9b90a5de56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:00.757101 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 909/1839] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:00.760272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5579138-c0e4-4285-9ed7-081310fb9185 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.760726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5579138-c0e4-4285-9ed7-081310fb9185 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.761122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5579138-c0e4-4285-9ed7-081310fb9185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.775440 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81665d-0d1b-489b-856f-c9b90a5de56b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:00.783518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7d5b6-2a03-47dd-bb9c-17f1eb47e471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:00.784308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7d5b6-2a03-47dd-bb9c-17f1eb47e471 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.784586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7d5b6-2a03-47dd-bb9c-17f1eb47e471 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.784868 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7d5b6-2a03-47dd-bb9c-17f1eb47e471 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.785583 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7d5b6-2a03-47dd-bb9c-17f1eb47e471 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:00.787505 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7d5b6-2a03-47dd-bb9c-17f1eb47e471 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:00.789188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-a5579138-c0e4-4285-9ed7-081310fb9185 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:00.791889 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a5579138-c0e4-4285-9ed7-081310fb9185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:00.800631 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 910/1840] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:00.804982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5579138-c0e4-4285-9ed7-081310fb9185 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:00.815364 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5579138-c0e4-4285-9ed7-081310fb9185 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:00.818264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 931/1841] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:00 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:53:00.827575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7d5b6-2a03-47dd-bb9c-17f1eb47e471 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:00.845304 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:00.866925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:00.887672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba2e1f-e6a8-4478-97d4-42f757a361d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:00.888291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba2e1f-e6a8-4478-97d4-42f757a361d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.888580 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba2e1f-e6a8-4478-97d4-42f757a361d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.888855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba2e1f-e6a8-4478-97d4-42f757a361d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.889600 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba2e1f-e6a8-4478-97d4-42f757a361d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:53:00.889910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba2e1f-e6a8-4478-97d4-42f757a361d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:00.891602 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba2e1f-e6a8-4478-97d4-42f757a361d8 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:00.896420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7d5b6-2a03-47dd-bb9c-17f1eb47e471 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:00.901271 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ef31f-7f54-4c01-8b86-ea070cf7a7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:00.902013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ef31f-7f54-4c01-8b86-ea070cf7a7f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.902297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ef31f-7f54-4c01-8b86-ea070cf7a7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.902569 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ef31f-7f54-4c01-8b86-ea070cf7a7f4 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.903272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ef31f-7f54-4c01-8b86-ea070cf7a7f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:00.904050 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 911/1842] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:53:00.905234 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ef31f-7f54-4c01-8b86-ea070cf7a7f4 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:00.908276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba2e1f-e6a8-4478-97d4-42f757a361d8 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:00.919534 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba2e1f-e6a8-4478-97d4-42f757a361d8 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:00.924631 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b1ad7-d0df-4170-9af0-042a2bc68279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:00.925291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b1ad7-d0df-4170-9af0-042a2bc68279 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.925574 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b1ad7-d0df-4170-9af0-042a2bc68279 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.925847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b1ad7-d0df-4170-9af0-042a2bc68279 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.929384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 932/1843] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:00 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:00.929511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b1ad7-d0df-4170-9af0-042a2bc68279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:00.929511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b1ad7-d0df-4170-9af0-042a2bc68279 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:00.930528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ef31f-7f54-4c01-8b86-ea070cf7a7f4 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:00.941040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ef31f-7f54-4c01-8b86-ea070cf7a7f4 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:00.946178 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3eef5e-c721-48d6-98ae-6d40352a6ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:00.946860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3eef5e-c721-48d6-98ae-6d40352a6ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:00.947195 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3eef5e-c721-48d6-98ae-6d40352a6ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:00.947552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3eef5e-c721-48d6-98ae-6d40352a6ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:00.948503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3eef5e-c721-48d6-98ae-6d40352a6ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:00.950398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3eef5e-c721-48d6-98ae-6d40352a6ba8 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:00.957787 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 912/1844] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:00 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:53:00.960757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 933/1845] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:53:00.962893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b1ad7-d0df-4170-9af0-042a2bc68279 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:00.965317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3eef5e-c721-48d6-98ae-6d40352a6ba8 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:00.976277 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3eef5e-c721-48d6-98ae-6d40352a6ba8 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:01.046477 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4f083-ee2a-4e03-ab11-071841969e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:01.055804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b1ad7-d0df-4170-9af0-042a2bc68279 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:01.064189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4f083-ee2a-4e03-ab11-071841969e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:01.064633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75d124-ac65-4d3f-9025-6e340c7af0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:01.064864 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4f083-ee2a-4e03-ab11-071841969e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:01.065308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4f083-ee2a-4e03-ab11-071841969e8e admin admin] PATH_INFO: `/v3/projects/fa2700a96f5f424fa475b38ebeb55518/users/bc92ade9f7ca408394d23ee1855a83dc/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:01.065910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75d124-ac65-4d3f-9025-6e340c7af0aa admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:01.066299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75d124-ac65-4d3f-9025-6e340c7af0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:01.066678 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75d124-ac65-4d3f-9025-6e340c7af0aa admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:01.067743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75d124-ac65-4d3f-9025-6e340c7af0aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:01.070406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75d124-ac65-4d3f-9025-6e340c7af0aa admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:01.095069 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4f083-ee2a-4e03-ab11-071841969e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2700a96f5f424fa475b38ebeb55518, user_id=bc92ade9f7ca408394d23ee1855a83dc, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:01.097794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4f083-ee2a-4e03-ab11-071841969e8e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:01.103762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 913/1846] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:00 2022] POST /identity/v3/users => generated 284 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:53:01.106148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75d124-ac65-4d3f-9025-6e340c7af0aa admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:01.113786 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 934/1847] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:00 2022] PUT /identity/v3/projects/fa2700a96f5f424fa475b38ebeb55518/users/bc92ade9f7ca408394d23ee1855a83dc/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:53:01.119033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4f083-ee2a-4e03-ab11-071841969e8e admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:01.122100 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75d124-ac65-4d3f-9025-6e340c7af0aa admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:01.211255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4f083-ee2a-4e03-ab11-071841969e8e admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:01.220046 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe41e87-3454-428a-8462-700761f1365c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:01.221302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe41e87-3454-428a-8462-700761f1365c admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:01.221827 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe41e87-3454-428a-8462-700761f1365c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:01.222351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe41e87-3454-428a-8462-700761f1365c admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:01.223879 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe41e87-3454-428a-8462-700761f1365c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:01.227098 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe41e87-3454-428a-8462-700761f1365c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:01.247510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 935/1848] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:01.247702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d29ab3-ebf3-48a3-80fc-554c07a4dea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:01.247702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d29ab3-ebf3-48a3-80fc-554c07a4dea5 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:01.247702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d29ab3-ebf3-48a3-80fc-554c07a4dea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:01.247702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d29ab3-ebf3-48a3-80fc-554c07a4dea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:01.248029 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d29ab3-ebf3-48a3-80fc-554c07a4dea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:01.249099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe41e87-3454-428a-8462-700761f1365c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:01.249518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d29ab3-ebf3-48a3-80fc-554c07a4dea5 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:01.260951 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 914/1849] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:01.261859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe41e87-3454-428a-8462-700761f1365c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:01.265174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d29ab3-ebf3-48a3-80fc-554c07a4dea5 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:01.267853 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf4e4c-6f97-4d17-b692-98057d1eaed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:01.268827 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf4e4c-6f97-4d17-b692-98057d1eaed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:01.269181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf4e4c-6f97-4d17-b692-98057d1eaed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:01.269526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf4e4c-6f97-4d17-b692-98057d1eaed8 admin admin] PATH_INFO: `/v3/projects/fa2700a96f5f424fa475b38ebeb55518/users/bc92ade9f7ca408394d23ee1855a83dc/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:01.272200 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf4e4c-6f97-4d17-b692-98057d1eaed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2700a96f5f424fa475b38ebeb55518, user_id=bc92ade9f7ca408394d23ee1855a83dc, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:01.274529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf4e4c-6f97-4d17-b692-98057d1eaed8 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:01.277320 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d29ab3-ebf3-48a3-80fc-554c07a4dea5 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:01.282032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68204c-ffbd-45c6-b9c0-e015ee323251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:01.282972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68204c-ffbd-45c6-b9c0-e015ee323251 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:01.283306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:01.283413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68204c-ffbd-45c6-b9c0-e015ee323251 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:01.283826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68204c-ffbd-45c6-b9c0-e015ee323251 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:01.285041 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68204c-ffbd-45c6-b9c0-e015ee323251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:01.286512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68204c-ffbd-45c6-b9c0-e015ee323251 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:01.290145 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 936/1850] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:01 2022] PUT /identity/v3/projects/fa2700a96f5f424fa475b38ebeb55518/users/bc92ade9f7ca408394d23ee1855a83dc/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:53:01.304338 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 915/1851] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:53:01.304530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f48519-8e0d-41f4-a1f9-0ce6893502c9 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:01.304530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f48519-8e0d-41f4-a1f9-0ce6893502c9 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:01.304530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f48519-8e0d-41f4-a1f9-0ce6893502c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:01.306318 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68204c-ffbd-45c6-b9c0-e015ee323251 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:01.335544 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68204c-ffbd-45c6-b9c0-e015ee323251 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:01.359816 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-a7f48519-8e0d-41f4-a1f9-0ce6893502c9 None None] MFA Rules not processed for user `bc92ade9f7ca408394d23ee1855a83dc`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:01.361820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a7f48519-8e0d-41f4-a1f9-0ce6893502c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:01.374993 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:01.428901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e5de2-31fb-4567-9901-6354088455e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:01.430292 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e5de2-31fb-4567-9901-6354088455e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:01.430727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e5de2-31fb-4567-9901-6354088455e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:01.431293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e5de2-31fb-4567-9901-6354088455e6 admin admin] PATH_INFO: `/v3/projects/9399eecb76f24fcb83ad9a8195eed789/users/dd198cfbea954cb282cec02720fb6fad/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:01.453326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 937/1852] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:01.453546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e5de2-31fb-4567-9901-6354088455e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9399eecb76f24fcb83ad9a8195eed789, user_id=dd198cfbea954cb282cec02720fb6fad, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:01.455511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e5de2-31fb-4567-9901-6354088455e6 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:01.463294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f48519-8e0d-41f4-a1f9-0ce6893502c9 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:01.464556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a7f48519-8e0d-41f4-a1f9-0ce6893502c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:01.494774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 916/1853] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:01 2022] PUT /identity/v3/projects/9399eecb76f24fcb83ad9a8195eed789/users/dd198cfbea954cb282cec02720fb6fad/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:01.507445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a14aa8-b26d-4b95-bea1-d64601aca904 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:01.507892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a14aa8-b26d-4b95-bea1-d64601aca904 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:01.508250 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a14aa8-b26d-4b95-bea1-d64601aca904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:01.563467 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d2a14aa8-b26d-4b95-bea1-d64601aca904 None None] MFA Rules not processed for user `dd198cfbea954cb282cec02720fb6fad`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:01.564925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d2a14aa8-b26d-4b95-bea1-d64601aca904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:01.601474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f48519-8e0d-41f4-a1f9-0ce6893502c9 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:01.662960 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a9a83-aa1b-4a94-be2f-8c6e48f84e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:01.663627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a9a83-aa1b-4a94-be2f-8c6e48f84e80 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:01.663967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a9a83-aa1b-4a94-be2f-8c6e48f84e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:01.664248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a9a83-aa1b-4a94-be2f-8c6e48f84e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:01.668890 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 917/1854] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:01.673423 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a14aa8-b26d-4b95-bea1-d64601aca904 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:01.749738 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:01.756750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a14aa8-b26d-4b95-bea1-d64601aca904 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:01.763251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21783a-5731-49b5-ad50-22f3895f6cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:01.763966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21783a-5731-49b5-ad50-22f3895f6cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:01.764277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21783a-5731-49b5-ad50-22f3895f6cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:01.764557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21783a-5731-49b5-ad50-22f3895f6cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:01.765936 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21783a-5731-49b5-ad50-22f3895f6cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:01.767846 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a9a83-aa1b-4a94-be2f-8c6e48f84e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:01.767959 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21783a-5731-49b5-ad50-22f3895f6cb6 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:01.769896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a9a83-aa1b-4a94-be2f-8c6e48f84e80 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:01.781642 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 938/1855] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:01.810583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 918/1856] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:01 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:53:01.822907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a9a83-aa1b-4a94-be2f-8c6e48f84e80 service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:01.847448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a9a83-aa1b-4a94-be2f-8c6e48f84e80 service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:01.931326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21783a-5731-49b5-ad50-22f3895f6cb6 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:01.957055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d13e7-eb90-4cc6-babf-4012c3cd91ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:01.957776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d13e7-eb90-4cc6-babf-4012c3cd91ed admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:01.958117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d13e7-eb90-4cc6-babf-4012c3cd91ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:01.958398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d13e7-eb90-4cc6-babf-4012c3cd91ed admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:01.959090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d13e7-eb90-4cc6-babf-4012c3cd91ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:01.961161 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d13e7-eb90-4cc6-babf-4012c3cd91ed admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:01.991343 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 939/1857] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:01 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:01.993154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:02.002672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d13e7-eb90-4cc6-babf-4012c3cd91ed admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:02.019372 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d13e7-eb90-4cc6-babf-4012c3cd91ed admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:02.026549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633847f0-8847-4363-9be4-a761f0fc3bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:02.027334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633847f0-8847-4363-9be4-a761f0fc3bcb admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:02.027689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633847f0-8847-4363-9be4-a761f0fc3bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:02.029122 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633847f0-8847-4363-9be4-a761f0fc3bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:02.029914 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633847f0-8847-4363-9be4-a761f0fc3bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:02.031697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633847f0-8847-4363-9be4-a761f0fc3bcb admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:02.047436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21783a-5731-49b5-ad50-22f3895f6cb6 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:02.052965 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94896556-9311-499b-8699-c252ebb64ced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:02.053692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94896556-9311-499b-8699-c252ebb64ced service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:02.053910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 940/1858] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:02.054058 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94896556-9311-499b-8699-c252ebb64ced service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:02.054254 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94896556-9311-499b-8699-c252ebb64ced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:02.054715 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633847f0-8847-4363-9be4-a761f0fc3bcb admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:02.065376 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633847f0-8847-4363-9be4-a761f0fc3bcb admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:02.070424 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d3fd6-8cf8-4d4a-a28a-12ca69017c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:02.075347 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d3fd6-8cf8-4d4a-a28a-12ca69017c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:02.075757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d3fd6-8cf8-4d4a-a28a-12ca69017c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:02.076011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d3fd6-8cf8-4d4a-a28a-12ca69017c05 admin admin] PATH_INFO: `/v3/projects/2af28709aa024960affe6ab1e58c3b96/users/b437c1b4d91a4e13b9aaaa4d2bc3b296/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:02.111666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d3fd6-8cf8-4d4a-a28a-12ca69017c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af28709aa024960affe6ab1e58c3b96, user_id=b437c1b4d91a4e13b9aaaa4d2bc3b296, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:02.116917 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d3fd6-8cf8-4d4a-a28a-12ca69017c05 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:02.130081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94896556-9311-499b-8699-c252ebb64ced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:02.132177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94896556-9311-499b-8699-c252ebb64ced service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:02.148699 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 919/1859] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:02.153156 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 941/1860] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:02 2022] PUT /identity/v3/projects/2af28709aa024960affe6ab1e58c3b96/users/b437c1b4d91a4e13b9aaaa4d2bc3b296/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:02.158640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94896556-9311-499b-8699-c252ebb64ced service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:02.177135 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94896556-9311-499b-8699-c252ebb64ced service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:02.246535 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a909dd-9283-478a-9cbe-c1cb5304e3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:02.247221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a909dd-9283-478a-9cbe-c1cb5304e3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:02.247506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a909dd-9283-478a-9cbe-c1cb5304e3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:02.247834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a909dd-9283-478a-9cbe-c1cb5304e3a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:02.248579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a909dd-9283-478a-9cbe-c1cb5304e3a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:02.250206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a909dd-9283-478a-9cbe-c1cb5304e3a0 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:02.264905 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 920/1861] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:02.276879 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d3fd6-8cf8-4d4a-a28a-12ca69017c05 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:02.277566 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d3fd6-8cf8-4d4a-a28a-12ca69017c05 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:02.282311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b18a19-7054-4c35-b388-91c7b2522154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:02.283233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b18a19-7054-4c35-b388-91c7b2522154 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:02.283519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b18a19-7054-4c35-b388-91c7b2522154 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:02.283831 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b18a19-7054-4c35-b388-91c7b2522154 admin admin] PATH_INFO: `/v3/projects/2af28709aa024960affe6ab1e58c3b96/users/b437c1b4d91a4e13b9aaaa4d2bc3b296/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:02.285845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b18a19-7054-4c35-b388-91c7b2522154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af28709aa024960affe6ab1e58c3b96, user_id=b437c1b4d91a4e13b9aaaa4d2bc3b296, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:02.288689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b18a19-7054-4c35-b388-91c7b2522154 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:02.299483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 942/1862] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:02 2022] PUT /identity/v3/projects/2af28709aa024960affe6ab1e58c3b96/users/b437c1b4d91a4e13b9aaaa4d2bc3b296/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:53:02.309929 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409fe23-f8b5-48f5-8409-236771262cbe None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:02.310220 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409fe23-f8b5-48f5-8409-236771262cbe None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:02.310502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409fe23-f8b5-48f5-8409-236771262cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:02.339890 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-4409fe23-f8b5-48f5-8409-236771262cbe None None] MFA Rules not processed for user `b437c1b4d91a4e13b9aaaa4d2bc3b296`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:02.341100 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-4409fe23-f8b5-48f5-8409-236771262cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:02.421071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:02.428010 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 921/1863] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:02.440248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b18a19-7054-4c35-b388-91c7b2522154 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:02.442130 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e5b18a19-7054-4c35-b388-91c7b2522154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:02.530305 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b18a19-7054-4c35-b388-91c7b2522154 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:02.534531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90884285-e0ba-44c7-84ce-df01341d0cb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:02.535444 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90884285-e0ba-44c7-84ce-df01341d0cb6 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:02.536007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90884285-e0ba-44c7-84ce-df01341d0cb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:02.536420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90884285-e0ba-44c7-84ce-df01341d0cb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:02.547213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90884285-e0ba-44c7-84ce-df01341d0cb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:02.548750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90884285-e0ba-44c7-84ce-df01341d0cb6 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:02.559771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 943/1864] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:02 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:02.623802 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409fe23-f8b5-48f5-8409-236771262cbe None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:02.639127 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409fe23-f8b5-48f5-8409-236771262cbe None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:02.643743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad46e57-9f6e-4962-8921-74cffda9a8eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:02.644371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad46e57-9f6e-4962-8921-74cffda9a8eb service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:02.644653 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad46e57-9f6e-4962-8921-74cffda9a8eb service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:02.645225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad46e57-9f6e-4962-8921-74cffda9a8eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:02.727411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad46e57-9f6e-4962-8921-74cffda9a8eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:02.729159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad46e57-9f6e-4962-8921-74cffda9a8eb service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:02.740150 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 922/1865] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:06.173323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90884285-e0ba-44c7-84ce-df01341d0cb6 service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:06.224807 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:06.258305 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90884285-e0ba-44c7-84ce-df01341d0cb6 service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:06.263496 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce89a14-2ee9-4069-a26a-cd60d2fd5f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:06.264385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce89a14-2ee9-4069-a26a-cd60d2fd5f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:06.264668 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce89a14-2ee9-4069-a26a-cd60d2fd5f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:06.264978 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce89a14-2ee9-4069-a26a-cd60d2fd5f5d admin admin] PATH_INFO: `/v3/users/dd198cfbea954cb282cec02720fb6fad` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:06.266499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce89a14-2ee9-4069-a26a-cd60d2fd5f5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd198cfbea954cb282cec02720fb6fad)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:06.268501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce89a14-2ee9-4069-a26a-cd60d2fd5f5d admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:06.318561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-2ce89a14-2ee9-4069-a26a-cd60d2fd5f5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd198cfbea954cb282cec02720fb6fad'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:06.319262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-2ce89a14-2ee9-4069-a26a-cd60d2fd5f5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd198cfbea954cb282cec02720fb6fad'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:06.336461 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-2ce89a14-2ee9-4069-a26a-cd60d2fd5f5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd198cfbea954cb282cec02720fb6fad'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:06.349225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-2ce89a14-2ee9-4069-a26a-cd60d2fd5f5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd198cfbea954cb282cec02720fb6fad'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:06.366031 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-2ce89a14-2ee9-4069-a26a-cd60d2fd5f5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd198cfbea954cb282cec02720fb6fad'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:06.373628 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad46e57-9f6e-4962-8921-74cffda9a8eb service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:06.374597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 944/1866] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:06 2022] DELETE /identity/v3/users/dd198cfbea954cb282cec02720fb6fad => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:06.379635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd83bd5-5a92-4685-afb3-fdb1d04c8601 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:06.379957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd83bd5-5a92-4685-afb3-fdb1d04c8601 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:06.380242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd83bd5-5a92-4685-afb3-fdb1d04c8601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:06.389926 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad46e57-9f6e-4962-8921-74cffda9a8eb service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:06.412306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-bbd83bd5-5a92-4685-afb3-fdb1d04c8601 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:06.414053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-bbd83bd5-5a92-4685-afb3-fdb1d04c8601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:06.468535 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4266ad-67c4-492f-90b4-22040b3eaa46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:06.469194 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4266ad-67c4-492f-90b4-22040b3eaa46 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:06.469488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4266ad-67c4-492f-90b4-22040b3eaa46 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:06.469760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4266ad-67c4-492f-90b4-22040b3eaa46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:06.471578 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5f4266ad-67c4-492f-90b4-22040b3eaa46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:06.494268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 945/1867] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:06.505130 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd83bd5-5a92-4685-afb3-fdb1d04c8601 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:06.506680 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-bbd83bd5-5a92-4685-afb3-fdb1d04c8601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:06.552654 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4266ad-67c4-492f-90b4-22040b3eaa46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:06.554485 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4266ad-67c4-492f-90b4-22040b3eaa46 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:06.567107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 923/1868] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:06.570221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd83bd5-5a92-4685-afb3-fdb1d04c8601 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:06.574693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3e682-298a-469d-920e-098b66cea78c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:06.579934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3e682-298a-469d-920e-098b66cea78c service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:06.580658 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3e682-298a-469d-920e-098b66cea78c service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:06.581106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3e682-298a-469d-920e-098b66cea78c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:06.589888 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:06.593361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3e682-298a-469d-920e-098b66cea78c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:06.595198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3e682-298a-469d-920e-098b66cea78c service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:06.616968 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 946/1869] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:53:07.066863 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4266ad-67c4-492f-90b4-22040b3eaa46 service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:07.068383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5f4266ad-67c4-492f-90b4-22040b3eaa46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:07.093733 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4266ad-67c4-492f-90b4-22040b3eaa46 service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:07.103191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71981ac-02ba-4e0d-97d0-9919ecdf638e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:07.105193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71981ac-02ba-4e0d-97d0-9919ecdf638e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:07.105493 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71981ac-02ba-4e0d-97d0-9919ecdf638e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:07.105763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71981ac-02ba-4e0d-97d0-9919ecdf638e admin admin] PATH_INFO: `/v3/projects/9399eecb76f24fcb83ad9a8195eed789` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:07.107969 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71981ac-02ba-4e0d-97d0-9919ecdf638e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9399eecb76f24fcb83ad9a8195eed789)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:07.111676 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71981ac-02ba-4e0d-97d0-9919ecdf638e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:07.216614 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:07.221425 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b71981ac-02ba-4e0d-97d0-9919ecdf638e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9399eecb76f24fcb83ad9a8195eed789', 'request_id': 'req-b71981ac-02ba-4e0d-97d0-9919ecdf638e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:07.238590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b71981ac-02ba-4e0d-97d0-9919ecdf638e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9399eecb76f24fcb83ad9a8195eed789', 'request_id': 'req-b71981ac-02ba-4e0d-97d0-9919ecdf638e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:07.248797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b71981ac-02ba-4e0d-97d0-9919ecdf638e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9399eecb76f24fcb83ad9a8195eed789', 'request_id': 'req-b71981ac-02ba-4e0d-97d0-9919ecdf638e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:07.266245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b71981ac-02ba-4e0d-97d0-9919ecdf638e admin admin] The token cache is being invalidate because project 9399eecb76f24fcb83ad9a8195eed789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:07.276255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b71981ac-02ba-4e0d-97d0-9919ecdf638e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:07.276255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b71981ac-02ba-4e0d-97d0-9919ecdf638e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:07.276255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 924/1870] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:07 2022] DELETE /identity/v3/projects/9399eecb76f24fcb83ad9a8195eed789 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:07.278188 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3e682-298a-469d-920e-098b66cea78c service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:07.288688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3e682-298a-469d-920e-098b66cea78c service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:07.378814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbfdd4-4916-46b8-9c80-bf8f3aaf1da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:07.379616 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbfdd4-4916-46b8-9c80-bf8f3aaf1da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:07.379937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbfdd4-4916-46b8-9c80-bf8f3aaf1da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:07.380219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbfdd4-4916-46b8-9c80-bf8f3aaf1da5 admin admin] PATH_INFO: `/v3/users/b437c1b4d91a4e13b9aaaa4d2bc3b296` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:07.381463 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbfdd4-4916-46b8-9c80-bf8f3aaf1da5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b437c1b4d91a4e13b9aaaa4d2bc3b296)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:07.383402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbfdd4-4916-46b8-9c80-bf8f3aaf1da5 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:07.460565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e5fbfdd4-4916-46b8-9c80-bf8f3aaf1da5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b437c1b4d91a4e13b9aaaa4d2bc3b296'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:07.461651 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e5fbfdd4-4916-46b8-9c80-bf8f3aaf1da5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b437c1b4d91a4e13b9aaaa4d2bc3b296'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:07.494503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e5fbfdd4-4916-46b8-9c80-bf8f3aaf1da5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b437c1b4d91a4e13b9aaaa4d2bc3b296'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:07.504900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e5fbfdd4-4916-46b8-9c80-bf8f3aaf1da5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b437c1b4d91a4e13b9aaaa4d2bc3b296'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:07.518923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-e5fbfdd4-4916-46b8-9c80-bf8f3aaf1da5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b437c1b4d91a4e13b9aaaa4d2bc3b296'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:07.533713 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 947/1871] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:07 2022] DELETE /identity/v3/users/b437c1b4d91a4e13b9aaaa4d2bc3b296 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:07.879614 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71981ac-02ba-4e0d-97d0-9919ecdf638e admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:07.881168 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b71981ac-02ba-4e0d-97d0-9919ecdf638e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:07.897059 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71981ac-02ba-4e0d-97d0-9919ecdf638e admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:07.962472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031aba0-9c47-4c55-9f39-5823256ee9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:07.964521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031aba0-9c47-4c55-9f39-5823256ee9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:07.964984 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031aba0-9c47-4c55-9f39-5823256ee9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:07.965223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031aba0-9c47-4c55-9f39-5823256ee9e3 admin admin] PATH_INFO: `/v3/projects/2af28709aa024960affe6ab1e58c3b96` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:07.966684 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031aba0-9c47-4c55-9f39-5823256ee9e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af28709aa024960affe6ab1e58c3b96)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:07.969725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031aba0-9c47-4c55-9f39-5823256ee9e3 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:08.040609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-7031aba0-9c47-4c55-9f39-5823256ee9e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af28709aa024960affe6ab1e58c3b96', 'request_id': 'req-7031aba0-9c47-4c55-9f39-5823256ee9e3'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:08.051038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-7031aba0-9c47-4c55-9f39-5823256ee9e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af28709aa024960affe6ab1e58c3b96', 'request_id': 'req-7031aba0-9c47-4c55-9f39-5823256ee9e3'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:08.058809 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-7031aba0-9c47-4c55-9f39-5823256ee9e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af28709aa024960affe6ab1e58c3b96', 'request_id': 'req-7031aba0-9c47-4c55-9f39-5823256ee9e3'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:08.072468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-7031aba0-9c47-4c55-9f39-5823256ee9e3 admin admin] The token cache is being invalidate because project 2af28709aa024960affe6ab1e58c3b96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:08.072759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-7031aba0-9c47-4c55-9f39-5823256ee9e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:08.073459 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-7031aba0-9c47-4c55-9f39-5823256ee9e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:08.075438 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 925/1872] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:07 2022] DELETE /identity/v3/projects/2af28709aa024960affe6ab1e58c3b96 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:08.081117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1bb9b-076d-4d47-b01a-249963c605f9 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:08.081959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1bb9b-076d-4d47-b01a-249963c605f9 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:08.082470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1bb9b-076d-4d47-b01a-249963c605f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:08.116146 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-44d1bb9b-076d-4d47-b01a-249963c605f9 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:08.119266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-44d1bb9b-076d-4d47-b01a-249963c605f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:08.233025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031aba0-9c47-4c55-9f39-5823256ee9e3 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:08.236617 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7031aba0-9c47-4c55-9f39-5823256ee9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:08.262468 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 948/1873] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:08.270034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664ea2a-fec5-4454-bd05-6751e5589a41 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:08.270759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664ea2a-fec5-4454-bd05-6751e5589a41 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:08.271258 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664ea2a-fec5-4454-bd05-6751e5589a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:08.278887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031aba0-9c47-4c55-9f39-5823256ee9e3 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:08.288287 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e132b0-c008-4f0f-8f70-76ec7a1c95fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:08.291125 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e132b0-c008-4f0f-8f70-76ec7a1c95fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:08.291413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e132b0-c008-4f0f-8f70-76ec7a1c95fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:08.291692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e132b0-c008-4f0f-8f70-76ec7a1c95fd admin admin] PATH_INFO: `/v3/users/c493c37862ea4e86b47cde6c1c487a0f` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:08.293137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e132b0-c008-4f0f-8f70-76ec7a1c95fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c493c37862ea4e86b47cde6c1c487a0f)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:08.295290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e132b0-c008-4f0f-8f70-76ec7a1c95fd admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:08.306983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-5664ea2a-fec5-4454-bd05-6751e5589a41 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:08.308512 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-5664ea2a-fec5-4454-bd05-6751e5589a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:08.321765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 949/1874] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:08.326796 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664ea2a-fec5-4454-bd05-6751e5589a41 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:08.349397 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664ea2a-fec5-4454-bd05-6751e5589a41 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:08.366004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc682cc-5d9a-495a-b72f-0481430c5925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:08.366911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc682cc-5d9a-495a-b72f-0481430c5925 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:08.367295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc682cc-5d9a-495a-b72f-0481430c5925 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:08.367832 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc682cc-5d9a-495a-b72f-0481430c5925 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:08.369891 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc682cc-5d9a-495a-b72f-0481430c5925 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:53:08.370584 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc682cc-5d9a-495a-b72f-0481430c5925 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:08.372747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-15e132b0-c008-4f0f-8f70-76ec7a1c95fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c493c37862ea4e86b47cde6c1c487a0f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:08.372927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc682cc-5d9a-495a-b72f-0481430c5925 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:08.376169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:08.384199 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-15e132b0-c008-4f0f-8f70-76ec7a1c95fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c493c37862ea4e86b47cde6c1c487a0f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:08.385882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-15e132b0-c008-4f0f-8f70-76ec7a1c95fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c493c37862ea4e86b47cde6c1c487a0f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:08.388949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=78,keystone:DELETE=19,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:08.390664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 950/1875] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:53:08.395496 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc682cc-5d9a-495a-b72f-0481430c5925 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:08.404082 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-15e132b0-c008-4f0f-8f70-76ec7a1c95fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c493c37862ea4e86b47cde6c1c487a0f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:08.404984 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-15e132b0-c008-4f0f-8f70-76ec7a1c95fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c493c37862ea4e86b47cde6c1c487a0f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:08.405981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fcc682cc-5d9a-495a-b72f-0481430c5925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:08.424372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 926/1876] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:08 2022] DELETE /identity/v3/users/c493c37862ea4e86b47cde6c1c487a0f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:08.425488 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc682cc-5d9a-495a-b72f-0481430c5925 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:08.430453 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924676f1-1e57-4461-84a9-12203a134522 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:08.431652 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924676f1-1e57-4461-84a9-12203a134522 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:08.432159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924676f1-1e57-4461-84a9-12203a134522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:08.465771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-924676f1-1e57-4461-84a9-12203a134522 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:08.467408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-924676f1-1e57-4461-84a9-12203a134522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:08.530005 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36fab5-e506-4e2b-824d-07d016282580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:08.531337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36fab5-e506-4e2b-824d-07d016282580 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:08.531691 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36fab5-e506-4e2b-824d-07d016282580 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:08.532097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36fab5-e506-4e2b-824d-07d016282580 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:08.532876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36fab5-e506-4e2b-824d-07d016282580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:08.535103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36fab5-e506-4e2b-824d-07d016282580 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:08.581484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 927/1877] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:08.587262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 951/1878] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:08 2022] POST /identity/v3/projects => generated 367 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:08.590316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924676f1-1e57-4461-84a9-12203a134522 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:08.592272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-924676f1-1e57-4461-84a9-12203a134522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:08.594542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36fab5-e506-4e2b-824d-07d016282580 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:08.731897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924676f1-1e57-4461-84a9-12203a134522 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:08.732424 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36fab5-e506-4e2b-824d-07d016282580 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:08.745445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51442fa8-01e8-49a9-b637-a6a4ffa845c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:08.746967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51442fa8-01e8-49a9-b637-a6a4ffa845c6 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:08.747480 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51442fa8-01e8-49a9-b637-a6a4ffa845c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:08.748065 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51442fa8-01e8-49a9-b637-a6a4ffa845c6 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:08.749375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51442fa8-01e8-49a9-b637-a6a4ffa845c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:08.754068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51442fa8-01e8-49a9-b637-a6a4ffa845c6 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:08.804693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 952/1879] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:08 2022] POST /identity/v3/users => generated 284 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:08.809016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51442fa8-01e8-49a9-b637-a6a4ffa845c6 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:08.828138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51442fa8-01e8-49a9-b637-a6a4ffa845c6 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:08.836160 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d47c8-92a1-4a6a-bbc3-bd19c408e040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:08.837543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d47c8-92a1-4a6a-bbc3-bd19c408e040 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:08.837841 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d47c8-92a1-4a6a-bbc3-bd19c408e040 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:08.838175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d47c8-92a1-4a6a-bbc3-bd19c408e040 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:08.839015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d47c8-92a1-4a6a-bbc3-bd19c408e040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:08.841762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d47c8-92a1-4a6a-bbc3-bd19c408e040 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:08.842575 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d413ad7-dc92-4912-9cc3-c1aac78e75db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:08.844054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d413ad7-dc92-4912-9cc3-c1aac78e75db service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:08.844538 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d413ad7-dc92-4912-9cc3-c1aac78e75db service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:08.845010 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d413ad7-dc92-4912-9cc3-c1aac78e75db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:08.857931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d413ad7-dc92-4912-9cc3-c1aac78e75db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:08.862025 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 953/1880] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:53:08.864014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d413ad7-dc92-4912-9cc3-c1aac78e75db service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:08.865739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d47c8-92a1-4a6a-bbc3-bd19c408e040 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:08.878587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d47c8-92a1-4a6a-bbc3-bd19c408e040 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:08.880526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 928/1881] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:08.885776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeabdd2-100a-491e-9493-e91f1c8d00a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:08.888596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeabdd2-100a-491e-9493-e91f1c8d00a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:08.889226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeabdd2-100a-491e-9493-e91f1c8d00a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:08.889781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeabdd2-100a-491e-9493-e91f1c8d00a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:08.890757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeabdd2-100a-491e-9493-e91f1c8d00a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:08.893010 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeabdd2-100a-491e-9493-e91f1c8d00a6 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:08.907758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 954/1882] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:08.910819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d413ad7-dc92-4912-9cc3-c1aac78e75db service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:08.922086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d413ad7-dc92-4912-9cc3-c1aac78e75db service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:08.927617 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54d955-7f1c-464a-9461-b095c66c28bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:08.928725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54d955-7f1c-464a-9461-b095c66c28bf admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:08.929118 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54d955-7f1c-464a-9461-b095c66c28bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:08.929284 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54d955-7f1c-464a-9461-b095c66c28bf admin admin] PATH_INFO: `/v3/projects/dc851bc7d9194d50b83133e473992b4d/users/69fded38b1c44c33827732495be0447b/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:08.953754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54d955-7f1c-464a-9461-b095c66c28bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc851bc7d9194d50b83133e473992b4d, user_id=69fded38b1c44c33827732495be0447b, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:08.956833 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54d955-7f1c-464a-9461-b095c66c28bf admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:08.972275 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 929/1883] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:08 2022] PUT /identity/v3/projects/dc851bc7d9194d50b83133e473992b4d/users/69fded38b1c44c33827732495be0447b/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:53:08.984143 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c162782-d919-419f-a2ee-749dba979ab7 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:08.984428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c162782-d919-419f-a2ee-749dba979ab7 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:08.984709 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c162782-d919-419f-a2ee-749dba979ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:09.013280 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-1c162782-d919-419f-a2ee-749dba979ab7 None None] MFA Rules not processed for user `69fded38b1c44c33827732495be0447b`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:09.014523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1c162782-d919-419f-a2ee-749dba979ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:09.065928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:09.102956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 955/1884] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:08 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:09.107670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54d955-7f1c-464a-9461-b095c66c28bf admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:09.118208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:09.187071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54d955-7f1c-464a-9461-b095c66c28bf admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:09.192454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0da76-240e-474d-bd2c-6b5fd0d33e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:09.193146 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0da76-240e-474d-bd2c-6b5fd0d33e4a admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:09.193423 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0da76-240e-474d-bd2c-6b5fd0d33e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:09.193694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0da76-240e-474d-bd2c-6b5fd0d33e4a admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:09.194389 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0da76-240e-474d-bd2c-6b5fd0d33e4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:09.197230 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0da76-240e-474d-bd2c-6b5fd0d33e4a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:09.213957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c162782-d919-419f-a2ee-749dba979ab7 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:09.215465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1c162782-d919-419f-a2ee-749dba979ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:09.231434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c162782-d919-419f-a2ee-749dba979ab7 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:09.235805 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 930/1885] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:09 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:09.236752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64e6c3-b949-4657-9706-14e2aa28af73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:09.237647 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64e6c3-b949-4657-9706-14e2aa28af73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:09.237925 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64e6c3-b949-4657-9706-14e2aa28af73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:09.238204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64e6c3-b949-4657-9706-14e2aa28af73 admin admin] PATH_INFO: `/v3/projects/58ebed32cfe044668a7501caf7af6ef8` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:09.239904 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64e6c3-b949-4657-9706-14e2aa28af73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58ebed32cfe044668a7501caf7af6ef8)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:09.240099 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0da76-240e-474d-bd2c-6b5fd0d33e4a admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:09.241840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64e6c3-b949-4657-9706-14e2aa28af73 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:09.297142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8e64e6c3-b949-4657-9706-14e2aa28af73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58ebed32cfe044668a7501caf7af6ef8', 'request_id': 'req-8e64e6c3-b949-4657-9706-14e2aa28af73'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:09.304659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8e64e6c3-b949-4657-9706-14e2aa28af73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58ebed32cfe044668a7501caf7af6ef8', 'request_id': 'req-8e64e6c3-b949-4657-9706-14e2aa28af73'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:09.312542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8e64e6c3-b949-4657-9706-14e2aa28af73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58ebed32cfe044668a7501caf7af6ef8', 'request_id': 'req-8e64e6c3-b949-4657-9706-14e2aa28af73'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:09.314903 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0da76-240e-474d-bd2c-6b5fd0d33e4a admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:09.327255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8e64e6c3-b949-4657-9706-14e2aa28af73 admin admin] The token cache is being invalidate because project 58ebed32cfe044668a7501caf7af6ef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:09.327553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8e64e6c3-b949-4657-9706-14e2aa28af73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:09.328339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8e64e6c3-b949-4657-9706-14e2aa28af73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:09.329935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 956/1886] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:09 2022] DELETE /identity/v3/projects/58ebed32cfe044668a7501caf7af6ef8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:53:09.333896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64e6c3-b949-4657-9706-14e2aa28af73 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:09.335316 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8e64e6c3-b949-4657-9706-14e2aa28af73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:09.382872 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd94ccf-0325-4ffa-a256-2469b22cd03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:09.383658 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd94ccf-0325-4ffa-a256-2469b22cd03e admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:09.384760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd94ccf-0325-4ffa-a256-2469b22cd03e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:09.385118 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd94ccf-0325-4ffa-a256-2469b22cd03e admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:09.386139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd94ccf-0325-4ffa-a256-2469b22cd03e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:09.388431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd94ccf-0325-4ffa-a256-2469b22cd03e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:09.420126 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 931/1887] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:09 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:09.424243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64e6c3-b949-4657-9706-14e2aa28af73 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:09.427548 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd94ccf-0325-4ffa-a256-2469b22cd03e admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:09.429501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-dfd94ccf-0325-4ffa-a256-2469b22cd03e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:09.430021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793dca8-38a2-4550-9c02-c25d84293d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:09.430993 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793dca8-38a2-4550-9c02-c25d84293d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:09.431338 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793dca8-38a2-4550-9c02-c25d84293d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:09.431497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793dca8-38a2-4550-9c02-c25d84293d08 admin admin] PATH_INFO: `/v3/users/28e523eb8a774207a91d5630114f5cff` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:09.432891 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793dca8-38a2-4550-9c02-c25d84293d08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28e523eb8a774207a91d5630114f5cff)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:09.434823 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793dca8-38a2-4550-9c02-c25d84293d08 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:09.442432 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd94ccf-0325-4ffa-a256-2469b22cd03e admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:09.447868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e584e-a8eb-485a-abc7-43ef6f06f641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:09.448564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e584e-a8eb-485a-abc7-43ef6f06f641 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:09.448872 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e584e-a8eb-485a-abc7-43ef6f06f641 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:09.449120 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e584e-a8eb-485a-abc7-43ef6f06f641 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:09.449850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e584e-a8eb-485a-abc7-43ef6f06f641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:09.451466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e584e-a8eb-485a-abc7-43ef6f06f641 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:09.462530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 932/1888] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:09.466860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e584e-a8eb-485a-abc7-43ef6f06f641 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:09.481231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e584e-a8eb-485a-abc7-43ef6f06f641 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:09.488306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-3793dca8-38a2-4550-9c02-c25d84293d08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28e523eb8a774207a91d5630114f5cff'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:09.488931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a7031-4c7e-4fb8-be40-7eca00f466b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:09.489161 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-3793dca8-38a2-4550-9c02-c25d84293d08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28e523eb8a774207a91d5630114f5cff'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:09.490474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a7031-4c7e-4fb8-be40-7eca00f466b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:09.490986 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a7031-4c7e-4fb8-be40-7eca00f466b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:09.491490 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a7031-4c7e-4fb8-be40-7eca00f466b6 admin admin] PATH_INFO: `/v3/projects/f3d740ba7e7443e386aa9e1d6285c55d/users/13464f36946e4e1a8f6ff2b665f9bba7/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:09.500788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:09.502124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-3793dca8-38a2-4550-9c02-c25d84293d08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28e523eb8a774207a91d5630114f5cff'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:09.510427 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-3793dca8-38a2-4550-9c02-c25d84293d08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28e523eb8a774207a91d5630114f5cff'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:09.517189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a7031-4c7e-4fb8-be40-7eca00f466b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d740ba7e7443e386aa9e1d6285c55d, user_id=13464f36946e4e1a8f6ff2b665f9bba7, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:09.518268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-3793dca8-38a2-4550-9c02-c25d84293d08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28e523eb8a774207a91d5630114f5cff'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:09.519943 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a7031-4c7e-4fb8-be40-7eca00f466b6 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:09.524063 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:09.525575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 957/1889] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:09 2022] DELETE /identity/v3/users/28e523eb8a774207a91d5630114f5cff => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:53:09.542454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 933/1890] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:09 2022] PUT /identity/v3/projects/f3d740ba7e7443e386aa9e1d6285c55d/users/13464f36946e4e1a8f6ff2b665f9bba7/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:09.547941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793dca8-38a2-4550-9c02-c25d84293d08 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:09.549388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-3793dca8-38a2-4550-9c02-c25d84293d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:09.621549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793dca8-38a2-4550-9c02-c25d84293d08 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:09.687992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcde69-63ed-41fa-ba1e-3699f56b3e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:09.688709 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcde69-63ed-41fa-ba1e-3699f56b3e3f admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:09.688998 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcde69-63ed-41fa-ba1e-3699f56b3e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:09.689280 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcde69-63ed-41fa-ba1e-3699f56b3e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:09.690051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcde69-63ed-41fa-ba1e-3699f56b3e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:09.691734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcde69-63ed-41fa-ba1e-3699f56b3e3f admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:09.712666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 958/1891] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:09.714312 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a7031-4c7e-4fb8-be40-7eca00f466b6 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:09.727568 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a7031-4c7e-4fb8-be40-7eca00f466b6 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:09.732973 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28035cf5-91ae-420e-a6c0-676267fad8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:09.733820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28035cf5-91ae-420e-a6c0-676267fad8b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:09.734105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28035cf5-91ae-420e-a6c0-676267fad8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:09.734384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28035cf5-91ae-420e-a6c0-676267fad8b5 admin admin] PATH_INFO: `/v3/projects/f3d740ba7e7443e386aa9e1d6285c55d/users/13464f36946e4e1a8f6ff2b665f9bba7/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:09.736920 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28035cf5-91ae-420e-a6c0-676267fad8b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d740ba7e7443e386aa9e1d6285c55d, user_id=13464f36946e4e1a8f6ff2b665f9bba7, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:09.740349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28035cf5-91ae-420e-a6c0-676267fad8b5 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:09.754841 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 934/1892] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:09 2022] PUT /identity/v3/projects/f3d740ba7e7443e386aa9e1d6285c55d/users/13464f36946e4e1a8f6ff2b665f9bba7/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:53:09.766207 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08414d-c7b6-4769-9df7-645be4eb1d72 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:09.766493 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08414d-c7b6-4769-9df7-645be4eb1d72 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:09.766774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08414d-c7b6-4769-9df7-645be4eb1d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:09.798529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-fe08414d-c7b6-4769-9df7-645be4eb1d72 None None] MFA Rules not processed for user `13464f36946e4e1a8f6ff2b665f9bba7`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:09.800035 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fe08414d-c7b6-4769-9df7-645be4eb1d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:09.877037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28035cf5-91ae-420e-a6c0-676267fad8b5 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:09.879896 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-28035cf5-91ae-420e-a6c0-676267fad8b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:09.919952 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 959/1893] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:09 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:09.931822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08414d-c7b6-4769-9df7-645be4eb1d72 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:09.933530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fe08414d-c7b6-4769-9df7-645be4eb1d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:09.947909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:10.015617 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28035cf5-91ae-420e-a6c0-676267fad8b5 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:10.024138 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:10.026090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:10.027624 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:10.028153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670 admin admin] PATH_INFO: `/v3/projects/027b65cb8ab94c5eae06e15ef0e2f2a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:10.030057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=027b65cb8ab94c5eae06e15ef0e2f2a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:10.031430 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08414d-c7b6-4769-9df7-645be4eb1d72 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:10.032556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:10.035584 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88c03d-c2da-48e2-a06f-26f262722b4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:10.036320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88c03d-c2da-48e2-a06f-26f262722b4e service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:10.036612 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88c03d-c2da-48e2-a06f-26f262722b4e service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:10.036918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88c03d-c2da-48e2-a06f-26f262722b4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:10.062404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88c03d-c2da-48e2-a06f-26f262722b4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:10.065129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88c03d-c2da-48e2-a06f-26f262722b4e service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:10.078749 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 960/1894] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:09 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:10.111767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '027b65cb8ab94c5eae06e15ef0e2f2a1', 'request_id': 'req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:10.125602 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '027b65cb8ab94c5eae06e15ef0e2f2a1', 'request_id': 'req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:10.136256 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '027b65cb8ab94c5eae06e15ef0e2f2a1', 'request_id': 'req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:10.160325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670 admin admin] The token cache is being invalidate because project 027b65cb8ab94c5eae06e15ef0e2f2a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:10.160897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:10.161974 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:10.163840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 935/1895] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:09 2022] DELETE /identity/v3/projects/027b65cb8ab94c5eae06e15ef0e2f2a1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:10.172972 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319d0af-472b-4411-84df-127f1a0607fc None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:10.173292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319d0af-472b-4411-84df-127f1a0607fc None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:10.173571 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319d0af-472b-4411-84df-127f1a0607fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:10.180036 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:10.182589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:10.197515 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-4319d0af-472b-4411-84df-127f1a0607fc None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:10.199625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-4319d0af-472b-4411-84df-127f1a0607fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:10.246583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe5c06-0d2e-405e-bd89-87f18a1ca670 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:10.250238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd30100-9a4a-4d94-b119-601f5989e24a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:10.250912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd30100-9a4a-4d94-b119-601f5989e24a service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:10.251195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd30100-9a4a-4d94-b119-601f5989e24a service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:10.251475 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd30100-9a4a-4d94-b119-601f5989e24a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:10.253381 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1bd30100-9a4a-4d94-b119-601f5989e24a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:10.280173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 961/1896] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:10.283343 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:10.289776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f0ce5-5c04-4420-aaa7-1db79a080505 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:10.290898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f0ce5-5c04-4420-aaa7-1db79a080505 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:10.291136 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f0ce5-5c04-4420-aaa7-1db79a080505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:10.310203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-306f0ce5-5c04-4420-aaa7-1db79a080505 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:10.311454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-306f0ce5-5c04-4420-aaa7-1db79a080505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:10.325347 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd30100-9a4a-4d94-b119-601f5989e24a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:10.326960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd30100-9a4a-4d94-b119-601f5989e24a service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:10.328601 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 962/1897] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:10.333234 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f0ce5-5c04-4420-aaa7-1db79a080505 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:10.338854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 936/1898] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:10 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:53:10.345213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f0ce5-5c04-4420-aaa7-1db79a080505 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:10.350023 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6805b-8757-4d8b-b500-c029c83d8be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:10.350720 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6805b-8757-4d8b-b500-c029c83d8be3 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:10.351006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6805b-8757-4d8b-b500-c029c83d8be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:10.351287 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6805b-8757-4d8b-b500-c029c83d8be3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:10.352063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6805b-8757-4d8b-b500-c029c83d8be3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:53:10.352371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6805b-8757-4d8b-b500-c029c83d8be3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:10.358619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6805b-8757-4d8b-b500-c029c83d8be3 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:10.372781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 963/1899] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:53:10.377923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd30100-9a4a-4d94-b119-601f5989e24a service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:10.394787 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd30100-9a4a-4d94-b119-601f5989e24a service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:10.406353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd90f74-2e27-4493-843a-1c8b818fc48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:10.407052 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd90f74-2e27-4493-843a-1c8b818fc48d admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:10.407352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd90f74-2e27-4493-843a-1c8b818fc48d admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:10.407625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd90f74-2e27-4493-843a-1c8b818fc48d admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:10.408367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd90f74-2e27-4493-843a-1c8b818fc48d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:10.410301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd90f74-2e27-4493-843a-1c8b818fc48d admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:10.447190 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 937/1900] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:10 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:10.451454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6805b-8757-4d8b-b500-c029c83d8be3 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:10.466448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6805b-8757-4d8b-b500-c029c83d8be3 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:10.537540 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afa7b6-aaf7-4fb2-8130-0b77bde825d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:10.538220 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afa7b6-aaf7-4fb2-8130-0b77bde825d5 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:10.538500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afa7b6-aaf7-4fb2-8130-0b77bde825d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:10.538794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afa7b6-aaf7-4fb2-8130-0b77bde825d5 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:10.539466 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7afa7b6-aaf7-4fb2-8130-0b77bde825d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:10.541686 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7afa7b6-aaf7-4fb2-8130-0b77bde825d5 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:10.564702 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 964/1901] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:10 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:10.568748 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd90f74-2e27-4493-843a-1c8b818fc48d admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:10.584377 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd90f74-2e27-4493-843a-1c8b818fc48d admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:10.590034 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512368f7-7565-46fd-a7f1-8b77f938e8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:10.590847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512368f7-7565-46fd-a7f1-8b77f938e8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:10.591161 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512368f7-7565-46fd-a7f1-8b77f938e8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:10.591514 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512368f7-7565-46fd-a7f1-8b77f938e8d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:10.592472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512368f7-7565-46fd-a7f1-8b77f938e8d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:10.594502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512368f7-7565-46fd-a7f1-8b77f938e8d6 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:10.607749 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 938/1902] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:10.621011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afa7b6-aaf7-4fb2-8130-0b77bde825d5 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:10.636883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afa7b6-aaf7-4fb2-8130-0b77bde825d5 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:10.643049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31c6b8-3794-4da2-b9c4-6fc4bedcce1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:10.643957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31c6b8-3794-4da2-b9c4-6fc4bedcce1f admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:10.644245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31c6b8-3794-4da2-b9c4-6fc4bedcce1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:10.644561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31c6b8-3794-4da2-b9c4-6fc4bedcce1f admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:10.645405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31c6b8-3794-4da2-b9c4-6fc4bedcce1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:10.647071 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31c6b8-3794-4da2-b9c4-6fc4bedcce1f admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:10.662502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 965/1903] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:53:10.664000 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512368f7-7565-46fd-a7f1-8b77f938e8d6 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:10.680616 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512368f7-7565-46fd-a7f1-8b77f938e8d6 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:10.688916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d43d7d-6121-4a01-b886-0c2742a4c053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:10.690116 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d43d7d-6121-4a01-b886-0c2742a4c053 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:10.690621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d43d7d-6121-4a01-b886-0c2742a4c053 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:10.691126 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d43d7d-6121-4a01-b886-0c2742a4c053 admin admin] PATH_INFO: `/v3/projects/71ba10aed4ee40debf46930ad5b47f4e/users/820764fa28f241fda13a262ca925fbe8/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:10.709191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d43d7d-6121-4a01-b886-0c2742a4c053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71ba10aed4ee40debf46930ad5b47f4e, user_id=820764fa28f241fda13a262ca925fbe8, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:10.712498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d43d7d-6121-4a01-b886-0c2742a4c053 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:10.734498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 939/1904] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:10 2022] PUT /identity/v3/projects/71ba10aed4ee40debf46930ad5b47f4e/users/820764fa28f241fda13a262ca925fbe8/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:10.745455 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08b830-68b0-4524-af6a-b7ece611d18b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:10.745747 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08b830-68b0-4524-af6a-b7ece611d18b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:10.746023 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08b830-68b0-4524-af6a-b7ece611d18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:10.765926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:10.783265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-5a08b830-68b0-4524-af6a-b7ece611d18b None None] MFA Rules not processed for user `820764fa28f241fda13a262ca925fbe8`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:10.783265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-5a08b830-68b0-4524-af6a-b7ece611d18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:10.850785 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 966/1905] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:10.863340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d43d7d-6121-4a01-b886-0c2742a4c053 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:10.925467 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d43d7d-6121-4a01-b886-0c2742a4c053 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:10.929225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0a167-71b0-4763-9d4a-2d36bbe350bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:10.929876 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0a167-71b0-4763-9d4a-2d36bbe350bd service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:10.930159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0a167-71b0-4763-9d4a-2d36bbe350bd service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:10.930434 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0a167-71b0-4763-9d4a-2d36bbe350bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:10.940241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0a167-71b0-4763-9d4a-2d36bbe350bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:10.941711 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0a167-71b0-4763-9d4a-2d36bbe350bd service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:10.951500 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 940/1906] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:14.242709 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7901506-ef95-4468-81fb-a5ad630f5096 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:14.243942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7901506-ef95-4468-81fb-a5ad630f5096 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:14.244896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7901506-ef95-4468-81fb-a5ad630f5096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:14.309215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-a7901506-ef95-4468-81fb-a5ad630f5096 None None] MFA Rules not processed for user `33aff48933844401b86eb808937e35c4`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:14.310588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a7901506-ef95-4468-81fb-a5ad630f5096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:14.392409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 967/1907] 10.222.0.73 () {58 vars in 1015 bytes} [Thu Jul 21 20:53:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:14.489989 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0a167-71b0-4763-9d4a-2d36bbe350bd service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:14.491618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c8e0a167-71b0-4763-9d4a-2d36bbe350bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:14.512562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7901506-ef95-4468-81fb-a5ad630f5096 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:14.514206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a7901506-ef95-4468-81fb-a5ad630f5096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:14.518921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0a167-71b0-4763-9d4a-2d36bbe350bd service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:14.523807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5e1c3-d6bb-488a-a9d8-d2680ef85081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:14.525605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5e1c3-d6bb-488a-a9d8-d2680ef85081 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:14.526039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5e1c3-d6bb-488a-a9d8-d2680ef85081 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:14.526327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5e1c3-d6bb-488a-a9d8-d2680ef85081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:14.528234 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7901506-ef95-4468-81fb-a5ad630f5096 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:14.533160 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b98ec-5557-4187-ae01-5e8638410e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:14.533939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b98ec-5557-4187-ae01-5e8638410e24 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:14.534227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b98ec-5557-4187-ae01-5e8638410e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:14.534630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b98ec-5557-4187-ae01-5e8638410e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:14.540893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5e1c3-d6bb-488a-a9d8-d2680ef85081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:14.540893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5e1c3-d6bb-488a-a9d8-d2680ef85081 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:14.558666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589b98ec-5557-4187-ae01-5e8638410e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:14.560935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589b98ec-5557-4187-ae01-5e8638410e24 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:14.566060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 941/1908] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:14.575869 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 968/1909] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:53:16.207339 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d20a6-2fc4-48ac-a17c-73b734eb7f3e None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:16.208517 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d20a6-2fc4-48ac-a17c-73b734eb7f3e None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:16.209127 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d20a6-2fc4-48ac-a17c-73b734eb7f3e None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:16.210768 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 942/1910] 10.222.0.60 () {54 vars in 921 bytes} [Thu Jul 21 20:53:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:53:16.224594 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75685b3c-1154-4907-8a83-12ea0cce35be None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:16.225594 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75685b3c-1154-4907-8a83-12ea0cce35be None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:16.225892 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75685b3c-1154-4907-8a83-12ea0cce35be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:16.281327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-75685b3c-1154-4907-8a83-12ea0cce35be None None] MFA Rules not processed for user `77812b356c974d2b80102217956169db`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:16.284522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-75685b3c-1154-4907-8a83-12ea0cce35be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:16.476746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 969/1911] 10.222.0.60 () {58 vars in 1018 bytes} [Thu Jul 21 20:53:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:16.684841 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d20a6-2fc4-48ac-a17c-73b734eb7f3e None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:16.695091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-282d20a6-2fc4-48ac-a17c-73b734eb7f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:16.812066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d20a6-2fc4-48ac-a17c-73b734eb7f3e None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:16.834558 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33630db0-a804-4f99-81de-f4885831f372 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:16.839752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33630db0-a804-4f99-81de-f4885831f372 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:16.840913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33630db0-a804-4f99-81de-f4885831f372 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:16.841214 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33630db0-a804-4f99-81de-f4885831f372 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:16.877926 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33630db0-a804-4f99-81de-f4885831f372 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:16.895680 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33630db0-a804-4f99-81de-f4885831f372 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:16.924799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 943/1912] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:16.941959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75685b3c-1154-4907-8a83-12ea0cce35be None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:16.943552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-75685b3c-1154-4907-8a83-12ea0cce35be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:17.036815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:17.150183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75685b3c-1154-4907-8a83-12ea0cce35be None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:17.169536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82133548-ec6a-47f3-95e3-2867c79e7b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:17.174041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82133548-ec6a-47f3-95e3-2867c79e7b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:17.174407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82133548-ec6a-47f3-95e3-2867c79e7b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:17.174694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82133548-ec6a-47f3-95e3-2867c79e7b33 admin admin] PATH_INFO: `/v3/users/067061404a7a412a80cee7ded0f94263` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:17.181398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82133548-ec6a-47f3-95e3-2867c79e7b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=067061404a7a412a80cee7ded0f94263)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:17.184356 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82133548-ec6a-47f3-95e3-2867c79e7b33 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:17.229789 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc663490-d3ac-4c7b-8ef2-d083525e054c None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:17.230559 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc663490-d3ac-4c7b-8ef2-d083525e054c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:17.230559 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc663490-d3ac-4c7b-8ef2-d083525e054c None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:17.232050 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 944/1913] 10.222.0.60 () {54 vars in 921 bytes} [Thu Jul 21 20:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 20:53:17.271816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416308de-ed00-48f3-8f91-755531c3fc9c None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:17.272108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416308de-ed00-48f3-8f91-755531c3fc9c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:17.272389 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416308de-ed00-48f3-8f91-755531c3fc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:17.325548 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-416308de-ed00-48f3-8f91-755531c3fc9c None None] MFA Rules not processed for user `77812b356c974d2b80102217956169db`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:17.329847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-416308de-ed00-48f3-8f91-755531c3fc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:17.332004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-82133548-ec6a-47f3-95e3-2867c79e7b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '067061404a7a412a80cee7ded0f94263'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:17.335689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-82133548-ec6a-47f3-95e3-2867c79e7b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '067061404a7a412a80cee7ded0f94263'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:17.370541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 945/1914] 10.222.0.60 () {58 vars in 1018 bytes} [Thu Jul 21 20:53:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:17.376905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-82133548-ec6a-47f3-95e3-2867c79e7b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '067061404a7a412a80cee7ded0f94263'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:17.402991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-82133548-ec6a-47f3-95e3-2867c79e7b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '067061404a7a412a80cee7ded0f94263'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:17.411366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-82133548-ec6a-47f3-95e3-2867c79e7b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '067061404a7a412a80cee7ded0f94263'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:17.431142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 970/1915] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:16 2022] DELETE /identity/v3/users/067061404a7a412a80cee7ded0f94263 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:17.439139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33607eed-4383-4653-836c-cdc123b0bdbc None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:17.439796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33607eed-4383-4653-836c-cdc123b0bdbc None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:17.440307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33607eed-4383-4653-836c-cdc123b0bdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:17.447974 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82133548-ec6a-47f3-95e3-2867c79e7b33 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:17.450598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-82133548-ec6a-47f3-95e3-2867c79e7b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:17.513748 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-33607eed-4383-4653-836c-cdc123b0bdbc None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:17.516019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-33607eed-4383-4653-836c-cdc123b0bdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:17.636898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:17.701653 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82133548-ec6a-47f3-95e3-2867c79e7b33 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:17.715061 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 946/1916] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:17.725167 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32a1ff-e690-419e-81c0-c9aa43277216 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:17.725830 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32a1ff-e690-419e-81c0-c9aa43277216 service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:17.726117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32a1ff-e690-419e-81c0-c9aa43277216 service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:17.726395 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32a1ff-e690-419e-81c0-c9aa43277216 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:17.749290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32a1ff-e690-419e-81c0-c9aa43277216 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:17.751416 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32a1ff-e690-419e-81c0-c9aa43277216 service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:17.768217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 971/1917] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:17.863838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33607eed-4383-4653-836c-cdc123b0bdbc None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:17.865925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-33607eed-4383-4653-836c-cdc123b0bdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:17.958856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33607eed-4383-4653-836c-cdc123b0bdbc None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:17.970963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a69e7-32ed-4721-b511-ed66d813afbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:17.972261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a69e7-32ed-4721-b511-ed66d813afbf service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:17.973339 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a69e7-32ed-4721-b511-ed66d813afbf service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:17.973906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a69e7-32ed-4721-b511-ed66d813afbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:17.990650 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a69e7-32ed-4721-b511-ed66d813afbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:17.994001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a69e7-32ed-4721-b511-ed66d813afbf service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:18.021938 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 947/1918] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:18.952038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32a1ff-e690-419e-81c0-c9aa43277216 service nova] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:18.954183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-da32a1ff-e690-419e-81c0-c9aa43277216 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:18.979017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32a1ff-e690-419e-81c0-c9aa43277216 service nova] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:18.985286 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f1c9c-34ed-43fd-96af-4e1ff3d60383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:18.986614 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f1c9c-34ed-43fd-96af-4e1ff3d60383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:18.987017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f1c9c-34ed-43fd-96af-4e1ff3d60383 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:18.987414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f1c9c-34ed-43fd-96af-4e1ff3d60383 admin admin] PATH_INFO: `/v3/projects/f570b02b4fac441eb4e7a5a7c38c9ba4` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:18.988881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f1c9c-34ed-43fd-96af-4e1ff3d60383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f570b02b4fac441eb4e7a5a7c38c9ba4)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:18.990880 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f1c9c-34ed-43fd-96af-4e1ff3d60383 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:19.030433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:19.059630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-868f1c9c-34ed-43fd-96af-4e1ff3d60383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f570b02b4fac441eb4e7a5a7c38c9ba4', 'request_id': 'req-868f1c9c-34ed-43fd-96af-4e1ff3d60383'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:19.066112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-868f1c9c-34ed-43fd-96af-4e1ff3d60383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f570b02b4fac441eb4e7a5a7c38c9ba4', 'request_id': 'req-868f1c9c-34ed-43fd-96af-4e1ff3d60383'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:19.074941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-868f1c9c-34ed-43fd-96af-4e1ff3d60383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f570b02b4fac441eb4e7a5a7c38c9ba4', 'request_id': 'req-868f1c9c-34ed-43fd-96af-4e1ff3d60383'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:19.093934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-868f1c9c-34ed-43fd-96af-4e1ff3d60383 admin admin] The token cache is being invalidate because project f570b02b4fac441eb4e7a5a7c38c9ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:19.094344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-868f1c9c-34ed-43fd-96af-4e1ff3d60383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:19.095224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-868f1c9c-34ed-43fd-96af-4e1ff3d60383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:19.096972 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 972/1919] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:18 2022] DELETE /identity/v3/projects/f570b02b4fac441eb4e7a5a7c38c9ba4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:19.100880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a69e7-32ed-4721-b511-ed66d813afbf service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:19.111964 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a69e7-32ed-4721-b511-ed66d813afbf service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:19.178517 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b07ca3-09aa-4e2e-9071-769ae8085150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:19.179399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b07ca3-09aa-4e2e-9071-769ae8085150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:19.179747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b07ca3-09aa-4e2e-9071-769ae8085150 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:19.180066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b07ca3-09aa-4e2e-9071-769ae8085150 admin admin] PATH_INFO: `/v3/users/bc92ade9f7ca408394d23ee1855a83dc` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:19.182334 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b07ca3-09aa-4e2e-9071-769ae8085150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc92ade9f7ca408394d23ee1855a83dc)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:19.184357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b07ca3-09aa-4e2e-9071-769ae8085150 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:19.245624 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-92b07ca3-09aa-4e2e-9071-769ae8085150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc92ade9f7ca408394d23ee1855a83dc'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:19.255060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-92b07ca3-09aa-4e2e-9071-769ae8085150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc92ade9f7ca408394d23ee1855a83dc'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:19.255706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-92b07ca3-09aa-4e2e-9071-769ae8085150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc92ade9f7ca408394d23ee1855a83dc'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:19.264549 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-92b07ca3-09aa-4e2e-9071-769ae8085150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc92ade9f7ca408394d23ee1855a83dc'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:19.265258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-92b07ca3-09aa-4e2e-9071-769ae8085150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc92ade9f7ca408394d23ee1855a83dc'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:19.287472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 948/1920] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:19 2022] DELETE /identity/v3/users/bc92ade9f7ca408394d23ee1855a83dc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:19.652144 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f1c9c-34ed-43fd-96af-4e1ff3d60383 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:19.653591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-868f1c9c-34ed-43fd-96af-4e1ff3d60383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:19.678640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f1c9c-34ed-43fd-96af-4e1ff3d60383 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:19.744779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdece23-270b-4b2e-b115-328dedcfa3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:19.745649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdece23-270b-4b2e-b115-328dedcfa3a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:19.745930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdece23-270b-4b2e-b115-328dedcfa3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:19.746204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdece23-270b-4b2e-b115-328dedcfa3a2 admin admin] PATH_INFO: `/v3/projects/fa2700a96f5f424fa475b38ebeb55518` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:19.747419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdece23-270b-4b2e-b115-328dedcfa3a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa2700a96f5f424fa475b38ebeb55518)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:19.749532 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdece23-270b-4b2e-b115-328dedcfa3a2 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:19.811373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-0bdece23-270b-4b2e-b115-328dedcfa3a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa2700a96f5f424fa475b38ebeb55518', 'request_id': 'req-0bdece23-270b-4b2e-b115-328dedcfa3a2'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:19.823570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-0bdece23-270b-4b2e-b115-328dedcfa3a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa2700a96f5f424fa475b38ebeb55518', 'request_id': 'req-0bdece23-270b-4b2e-b115-328dedcfa3a2'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:19.830109 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-0bdece23-270b-4b2e-b115-328dedcfa3a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa2700a96f5f424fa475b38ebeb55518', 'request_id': 'req-0bdece23-270b-4b2e-b115-328dedcfa3a2'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:19.844837 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-0bdece23-270b-4b2e-b115-328dedcfa3a2 admin admin] The token cache is being invalidate because project fa2700a96f5f424fa475b38ebeb55518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:19.845047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-0bdece23-270b-4b2e-b115-328dedcfa3a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:19.845693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-0bdece23-270b-4b2e-b115-328dedcfa3a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:19.847286 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 973/1921] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:19 2022] DELETE /identity/v3/projects/fa2700a96f5f424fa475b38ebeb55518 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:19.854067 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a656312-4ef1-4439-b273-697829add023 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:19.854344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a656312-4ef1-4439-b273-697829add023 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:19.854622 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a656312-4ef1-4439-b273-697829add023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:19.861651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:19.882972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-8a656312-4ef1-4439-b273-697829add023 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:19.885120 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8a656312-4ef1-4439-b273-697829add023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:19.963599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 949/1922] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:19.972617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fccfc0-5ced-4f8b-bc3e-ea567ef94dca None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:19.973561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fccfc0-5ced-4f8b-bc3e-ea567ef94dca None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:19.973913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fccfc0-5ced-4f8b-bc3e-ea567ef94dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:20.005617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-13fccfc0-5ced-4f8b-bc3e-ea567ef94dca None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:20.007181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-13fccfc0-5ced-4f8b-bc3e-ea567ef94dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:20.015939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 974/1923] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:20.020387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a656312-4ef1-4439-b273-697829add023 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:20.032200 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a656312-4ef1-4439-b273-697829add023 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:20.051967 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7322e-545c-4553-85d0-c5ffdbb10176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:20.053091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7322e-545c-4553-85d0-c5ffdbb10176 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:20.053552 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7322e-545c-4553-85d0-c5ffdbb10176 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:20.054005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7322e-545c-4553-85d0-c5ffdbb10176 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:20.059778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7322e-545c-4553-85d0-c5ffdbb10176 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:53:20.060201 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7322e-545c-4553-85d0-c5ffdbb10176 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:20.063098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7322e-545c-4553-85d0-c5ffdbb10176 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:20.085761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 950/1924] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:53:20.087570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fccfc0-5ced-4f8b-bc3e-ea567ef94dca None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:20.097842 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fccfc0-5ced-4f8b-bc3e-ea567ef94dca None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:20.103508 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f24415-0a6c-4be9-916d-16c527053b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:20.104432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f24415-0a6c-4be9-916d-16c527053b32 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:20.104819 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f24415-0a6c-4be9-916d-16c527053b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:20.105331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f24415-0a6c-4be9-916d-16c527053b32 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:20.106290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f24415-0a6c-4be9-916d-16c527053b32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:20.108354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f24415-0a6c-4be9-916d-16c527053b32 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:20.141751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 975/1925] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:20 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:20.152957 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7322e-545c-4553-85d0-c5ffdbb10176 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:20.168503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7322e-545c-4553-85d0-c5ffdbb10176 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:20.227907 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8ec6a-7ab7-4ff8-962c-00715597f24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:20.228592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8ec6a-7ab7-4ff8-962c-00715597f24a admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:20.228879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8ec6a-7ab7-4ff8-962c-00715597f24a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:20.229155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8ec6a-7ab7-4ff8-962c-00715597f24a admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:20.229855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8ec6a-7ab7-4ff8-962c-00715597f24a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:20.231854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8ec6a-7ab7-4ff8-962c-00715597f24a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:20.253609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 951/1926] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:20 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:20.260910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f24415-0a6c-4be9-916d-16c527053b32 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:20.271703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f24415-0a6c-4be9-916d-16c527053b32 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:20.277664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65feef65-ebcc-4bb6-8ba2-347cf48b2a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:20.278377 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65feef65-ebcc-4bb6-8ba2-347cf48b2a24 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:20.278661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65feef65-ebcc-4bb6-8ba2-347cf48b2a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:20.278939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65feef65-ebcc-4bb6-8ba2-347cf48b2a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:20.279678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65feef65-ebcc-4bb6-8ba2-347cf48b2a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:20.281309 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65feef65-ebcc-4bb6-8ba2-347cf48b2a24 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:20.292810 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 976/1927] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:20.296925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8ec6a-7ab7-4ff8-962c-00715597f24a admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:20.307331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8ec6a-7ab7-4ff8-962c-00715597f24a admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:20.312311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736d2bc-df8a-4212-8e37-366176f6fb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:20.312987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736d2bc-df8a-4212-8e37-366176f6fb77 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:20.313269 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736d2bc-df8a-4212-8e37-366176f6fb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:20.313543 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736d2bc-df8a-4212-8e37-366176f6fb77 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:20.314591 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736d2bc-df8a-4212-8e37-366176f6fb77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:20.315953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736d2bc-df8a-4212-8e37-366176f6fb77 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:20.327445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 952/1928] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:20.331913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65feef65-ebcc-4bb6-8ba2-347cf48b2a24 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:20.342816 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65feef65-ebcc-4bb6-8ba2-347cf48b2a24 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:20.347450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddc725-54b2-4daf-8741-591717604afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:20.348436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddc725-54b2-4daf-8741-591717604afe admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:20.348660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddc725-54b2-4daf-8741-591717604afe admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:20.348936 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddc725-54b2-4daf-8741-591717604afe admin admin] PATH_INFO: `/v3/projects/0be25e28ff934eadbaec30761c443d69/users/e09e2cd28e9a4fb99210416e1333903e/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:20.351881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:20.368505 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddc725-54b2-4daf-8741-591717604afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0be25e28ff934eadbaec30761c443d69, user_id=e09e2cd28e9a4fb99210416e1333903e, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:20.371863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddc725-54b2-4daf-8741-591717604afe admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:20.387486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 977/1929] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:20 2022] PUT /identity/v3/projects/0be25e28ff934eadbaec30761c443d69/users/e09e2cd28e9a4fb99210416e1333903e/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:20.398017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f36186-74bf-47b6-a53c-f171db7f9b22 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:20.398309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f36186-74bf-47b6-a53c-f171db7f9b22 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:20.398593 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f36186-74bf-47b6-a53c-f171db7f9b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:20.429088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-04f36186-74bf-47b6-a53c-f171db7f9b22 None None] MFA Rules not processed for user `e09e2cd28e9a4fb99210416e1333903e`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:20.430828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-04f36186-74bf-47b6-a53c-f171db7f9b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:20.495633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:20.502739 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 953/1930] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:20.508015 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddc725-54b2-4daf-8741-591717604afe admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:20.582615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddc725-54b2-4daf-8741-591717604afe admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:20.587471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffa7e5-4516-47d4-8c1e-ca0b746a9a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:20.588203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffa7e5-4516-47d4-8c1e-ca0b746a9a6b admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:20.588487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffa7e5-4516-47d4-8c1e-ca0b746a9a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:20.588762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffa7e5-4516-47d4-8c1e-ca0b746a9a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:20.589476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffa7e5-4516-47d4-8c1e-ca0b746a9a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:20.591407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffa7e5-4516-47d4-8c1e-ca0b746a9a6b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:20.621564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 978/1931] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:20 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:53:20.625455 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f36186-74bf-47b6-a53c-f171db7f9b22 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:20.637407 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f36186-74bf-47b6-a53c-f171db7f9b22 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:20.727568 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93d04d-040a-4234-8789-4536bbb26cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:20.728597 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93d04d-040a-4234-8789-4536bbb26cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:20.729090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93d04d-040a-4234-8789-4536bbb26cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:20.729450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93d04d-040a-4234-8789-4536bbb26cd9 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:20.730429 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93d04d-040a-4234-8789-4536bbb26cd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:20.732828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93d04d-040a-4234-8789-4536bbb26cd9 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:20.758343 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 954/1932] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:20 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:20.761251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffa7e5-4516-47d4-8c1e-ca0b746a9a6b admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:20.771768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffa7e5-4516-47d4-8c1e-ca0b746a9a6b admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:20.776797 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424681c4-b99e-4b70-a056-daff77b08ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:20.777511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424681c4-b99e-4b70-a056-daff77b08ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:20.777754 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424681c4-b99e-4b70-a056-daff77b08ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:20.778034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424681c4-b99e-4b70-a056-daff77b08ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:20.778769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424681c4-b99e-4b70-a056-daff77b08ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:20.780496 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424681c4-b99e-4b70-a056-daff77b08ac1 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:20.797834 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 979/1933] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:20.802056 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93d04d-040a-4234-8789-4536bbb26cd9 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:20.813750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93d04d-040a-4234-8789-4536bbb26cd9 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:20.818480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4543b75-b654-45c0-951e-e40c3855312f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:20.819153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4543b75-b654-45c0-951e-e40c3855312f admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:20.819431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4543b75-b654-45c0-951e-e40c3855312f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:20.819703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4543b75-b654-45c0-951e-e40c3855312f admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:20.820461 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4543b75-b654-45c0-951e-e40c3855312f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:20.822296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4543b75-b654-45c0-951e-e40c3855312f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:20.835754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 955/1934] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:20.838591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424681c4-b99e-4b70-a056-daff77b08ac1 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:20.849549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424681c4-b99e-4b70-a056-daff77b08ac1 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:20.855262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f9ea6-4603-4b01-a563-b1f704418123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:20.856206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f9ea6-4603-4b01-a563-b1f704418123 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:20.856488 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f9ea6-4603-4b01-a563-b1f704418123 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:20.856763 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f9ea6-4603-4b01-a563-b1f704418123 admin admin] PATH_INFO: `/v3/projects/e75bd814be1148dfb85d2103a2e4f6ab/users/3617708a44b44744bc49567d0fd97ad4/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:20.873076 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f9ea6-4603-4b01-a563-b1f704418123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e75bd814be1148dfb85d2103a2e4f6ab, user_id=3617708a44b44744bc49567d0fd97ad4, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:20.875620 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f9ea6-4603-4b01-a563-b1f704418123 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:20.889703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 980/1935] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:20 2022] PUT /identity/v3/projects/e75bd814be1148dfb85d2103a2e4f6ab/users/3617708a44b44744bc49567d0fd97ad4/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:20.901529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa19b2b-a7bc-461b-b4d5-b01f3b79673c None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:20.901890 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa19b2b-a7bc-461b-b4d5-b01f3b79673c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:20.902226 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa19b2b-a7bc-461b-b4d5-b01f3b79673c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:20.931662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-3fa19b2b-a7bc-461b-b4d5-b01f3b79673c None None] MFA Rules not processed for user `3617708a44b44744bc49567d0fd97ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:20.932940 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-3fa19b2b-a7bc-461b-b4d5-b01f3b79673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:21.025075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 956/1936] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:21.029618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f9ea6-4603-4b01-a563-b1f704418123 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:21.106570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f9ea6-4603-4b01-a563-b1f704418123 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:21.111962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79591d-badb-45ab-af1a-f4cf5e3dbaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:21.112640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79591d-badb-45ab-af1a-f4cf5e3dbaf2 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:21.112925 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79591d-badb-45ab-af1a-f4cf5e3dbaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:21.113200 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79591d-badb-45ab-af1a-f4cf5e3dbaf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:21.113914 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79591d-badb-45ab-af1a-f4cf5e3dbaf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:21.116100 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79591d-badb-45ab-af1a-f4cf5e3dbaf2 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:21.161109 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 981/1937] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:21 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:21.166359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa19b2b-a7bc-461b-b4d5-b01f3b79673c None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:21.178258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa19b2b-a7bc-461b-b4d5-b01f3b79673c None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:21.266484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f36b19-4731-4ddc-8bfb-4c20cf17c35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:21.267177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f36b19-4731-4ddc-8bfb-4c20cf17c35f admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:21.267634 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f36b19-4731-4ddc-8bfb-4c20cf17c35f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:21.267952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f36b19-4731-4ddc-8bfb-4c20cf17c35f admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:21.268664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f36b19-4731-4ddc-8bfb-4c20cf17c35f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:21.270607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f36b19-4731-4ddc-8bfb-4c20cf17c35f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:21.287761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:21.307432 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 957/1938] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:21 2022] POST /identity/v3/users => generated 295 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:53:21.308291 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79591d-badb-45ab-af1a-f4cf5e3dbaf2 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:21.321765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79591d-badb-45ab-af1a-f4cf5e3dbaf2 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:21.328859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a37ca-ac20-414e-b819-bd63e7ed7618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:21.330004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a37ca-ac20-414e-b819-bd63e7ed7618 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:21.330004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a37ca-ac20-414e-b819-bd63e7ed7618 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:21.330177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a37ca-ac20-414e-b819-bd63e7ed7618 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:21.330920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a37ca-ac20-414e-b819-bd63e7ed7618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:21.333400 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a37ca-ac20-414e-b819-bd63e7ed7618 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:21.349198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 982/1939] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:53:21.356620 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f36b19-4731-4ddc-8bfb-4c20cf17c35f admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:21.368287 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f36b19-4731-4ddc-8bfb-4c20cf17c35f admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:21.373481 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf9e64-c1b8-42c7-ae11-ac64a4a3605c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:21.374966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf9e64-c1b8-42c7-ae11-ac64a4a3605c admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:21.375434 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf9e64-c1b8-42c7-ae11-ac64a4a3605c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:21.375935 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf9e64-c1b8-42c7-ae11-ac64a4a3605c admin admin] PATH_INFO: `/v3/projects/b045112b7c4d47bb8735b8a352efda81/users/34f20989c7434905b4eb87d80fec8620/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:21.401470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf9e64-c1b8-42c7-ae11-ac64a4a3605c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b045112b7c4d47bb8735b8a352efda81, user_id=34f20989c7434905b4eb87d80fec8620, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:21.406349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf9e64-c1b8-42c7-ae11-ac64a4a3605c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:21.425200 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 958/1940] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:21 2022] PUT /identity/v3/projects/b045112b7c4d47bb8735b8a352efda81/users/34f20989c7434905b4eb87d80fec8620/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:21.429323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a37ca-ac20-414e-b819-bd63e7ed7618 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:21.439866 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a37ca-ac20-414e-b819-bd63e7ed7618 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:21.444228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:21.505843 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c88b4-6672-4128-8428-b6217f5be20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:21.506517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c88b4-6672-4128-8428-b6217f5be20e admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:21.506803 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c88b4-6672-4128-8428-b6217f5be20e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:21.507081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c88b4-6672-4128-8428-b6217f5be20e admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:21.507840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c88b4-6672-4128-8428-b6217f5be20e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:21.509530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c88b4-6672-4128-8428-b6217f5be20e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:21.519924 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 983/1941] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:21.524844 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf9e64-c1b8-42c7-ae11-ac64a4a3605c admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:21.536541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf9e64-c1b8-42c7-ae11-ac64a4a3605c admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:21.542387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e94db-20d9-4244-9686-c2c3d8a0bf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:21.543293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e94db-20d9-4244-9686-c2c3d8a0bf09 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:21.543577 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e94db-20d9-4244-9686-c2c3d8a0bf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:21.543886 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e94db-20d9-4244-9686-c2c3d8a0bf09 admin admin] PATH_INFO: `/v3/projects/b045112b7c4d47bb8735b8a352efda81/users/34f20989c7434905b4eb87d80fec8620/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:21.545988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e94db-20d9-4244-9686-c2c3d8a0bf09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b045112b7c4d47bb8735b8a352efda81, user_id=34f20989c7434905b4eb87d80fec8620, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:21.548698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e94db-20d9-4244-9686-c2c3d8a0bf09 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:21.560319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 959/1942] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:21 2022] PUT /identity/v3/projects/b045112b7c4d47bb8735b8a352efda81/users/34f20989c7434905b4eb87d80fec8620/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:21.571472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab373f-3496-4dcf-a919-315d836d7cb8 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:21.571867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab373f-3496-4dcf-a919-315d836d7cb8 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:21.572378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab373f-3496-4dcf-a919-315d836d7cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:21.602735 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-7bab373f-3496-4dcf-a919-315d836d7cb8 None None] MFA Rules not processed for user `34f20989c7434905b4eb87d80fec8620`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:21.604130 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-7bab373f-3496-4dcf-a919-315d836d7cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:21.676205 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 984/1943] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:21.688010 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e94db-20d9-4244-9686-c2c3d8a0bf09 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:21.690425 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-225e94db-20d9-4244-9686-c2c3d8a0bf09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:21.766218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e94db-20d9-4244-9686-c2c3d8a0bf09 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:21.773355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05ad5a-7a18-4f81-a7d9-8f67ff34a50b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:21.775191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05ad5a-7a18-4f81-a7d9-8f67ff34a50b service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:21.775692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05ad5a-7a18-4f81-a7d9-8f67ff34a50b service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:21.776195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05ad5a-7a18-4f81-a7d9-8f67ff34a50b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:21.792000 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05ad5a-7a18-4f81-a7d9-8f67ff34a50b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:21.794664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05ad5a-7a18-4f81-a7d9-8f67ff34a50b service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:21.812122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 960/1944] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:21.878794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab373f-3496-4dcf-a919-315d836d7cb8 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:21.895031 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab373f-3496-4dcf-a919-315d836d7cb8 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:21.900363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aedfaa-3d3e-4085-b90b-076981388085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:21.901298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aedfaa-3d3e-4085-b90b-076981388085 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:21.901745 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aedfaa-3d3e-4085-b90b-076981388085 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:21.902128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aedfaa-3d3e-4085-b90b-076981388085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:21.971383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aedfaa-3d3e-4085-b90b-076981388085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:21.972782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aedfaa-3d3e-4085-b90b-076981388085 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:21.982447 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 985/1945] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:25.112645 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05ad5a-7a18-4f81-a7d9-8f67ff34a50b service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:25.115022 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ff05ad5a-7a18-4f81-a7d9-8f67ff34a50b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:25.194482 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05ad5a-7a18-4f81-a7d9-8f67ff34a50b service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:25.200250 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3b2a6-1693-4d90-bea7-123ed855e7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:25.201176 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3b2a6-1693-4d90-bea7-123ed855e7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:25.201485 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3b2a6-1693-4d90-bea7-123ed855e7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:25.201820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3b2a6-1693-4d90-bea7-123ed855e7fa admin admin] PATH_INFO: `/v3/users/820764fa28f241fda13a262ca925fbe8` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:25.203678 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3b2a6-1693-4d90-bea7-123ed855e7fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=820764fa28f241fda13a262ca925fbe8)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:25.205767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3b2a6-1693-4d90-bea7-123ed855e7fa admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:25.259191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-def3b2a6-1693-4d90-bea7-123ed855e7fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '820764fa28f241fda13a262ca925fbe8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:25.267755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-def3b2a6-1693-4d90-bea7-123ed855e7fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '820764fa28f241fda13a262ca925fbe8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:25.268372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-def3b2a6-1693-4d90-bea7-123ed855e7fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '820764fa28f241fda13a262ca925fbe8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:25.276727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-def3b2a6-1693-4d90-bea7-123ed855e7fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '820764fa28f241fda13a262ca925fbe8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:25.277428 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-def3b2a6-1693-4d90-bea7-123ed855e7fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '820764fa28f241fda13a262ca925fbe8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:25.278720 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:25.299140 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 961/1946] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:25 2022] DELETE /identity/v3/users/820764fa28f241fda13a262ca925fbe8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:25.304430 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a70a6d-90d7-494a-9dc8-9be3e0f10232 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:25.304938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a70a6d-90d7-494a-9dc8-9be3e0f10232 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:25.305050 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a70a6d-90d7-494a-9dc8-9be3e0f10232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:25.324278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-05a70a6d-90d7-494a-9dc8-9be3e0f10232 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:25.325896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-05a70a6d-90d7-494a-9dc8-9be3e0f10232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:25.348153 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:25.392066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 986/1947] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:25.408423 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3b2a6-1693-4d90-bea7-123ed855e7fa admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:25.409892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-def3b2a6-1693-4d90-bea7-123ed855e7fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:25.496178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3b2a6-1693-4d90-bea7-123ed855e7fa admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:25.502272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b81901-c38b-4619-a38a-143d1a71407a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:25.503050 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b81901-c38b-4619-a38a-143d1a71407a service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:25.503363 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b81901-c38b-4619-a38a-143d1a71407a service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:25.503674 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b81901-c38b-4619-a38a-143d1a71407a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:25.515094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b81901-c38b-4619-a38a-143d1a71407a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:25.516907 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b81901-c38b-4619-a38a-143d1a71407a service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:25.534613 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 962/1948] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:25.873850 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a70a6d-90d7-494a-9dc8-9be3e0f10232 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:25.875689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-05a70a6d-90d7-494a-9dc8-9be3e0f10232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:25.892992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a70a6d-90d7-494a-9dc8-9be3e0f10232 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:25.901485 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:25.904234 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:25.904925 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:25.905473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501 admin admin] PATH_INFO: `/v3/projects/71ba10aed4ee40debf46930ad5b47f4e` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:25.907333 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71ba10aed4ee40debf46930ad5b47f4e)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:25.909906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:25.983158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71ba10aed4ee40debf46930ad5b47f4e', 'request_id': 'req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:25.991605 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71ba10aed4ee40debf46930ad5b47f4e', 'request_id': 'req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:25.999836 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71ba10aed4ee40debf46930ad5b47f4e', 'request_id': 'req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.017277 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501 admin admin] The token cache is being invalidate because project 71ba10aed4ee40debf46930ad5b47f4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:26.017908 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.018580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.022667 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 987/1949] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:25 2022] DELETE /identity/v3/projects/71ba10aed4ee40debf46930ad5b47f4e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:26.029090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df05817-a4ff-4646-aff8-1083598dfeb9 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:26.029405 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df05817-a4ff-4646-aff8-1083598dfeb9 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:26.029946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df05817-a4ff-4646-aff8-1083598dfeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:26.039600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:26.041718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:26.056807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-5df05817-a4ff-4646-aff8-1083598dfeb9 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:26.058301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5df05817-a4ff-4646-aff8-1083598dfeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:26.114969 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6cb5c-41d4-40a6-bcdf-c3810d355501 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:26.122546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b14cb-b5e1-4759-b085-77ec08210ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:26.123469 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b14cb-b5e1-4759-b085-77ec08210ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:26.123801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b14cb-b5e1-4759-b085-77ec08210ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:26.124061 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b14cb-b5e1-4759-b085-77ec08210ffe admin admin] PATH_INFO: `/v3/users/69fded38b1c44c33827732495be0447b` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:26.128649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b14cb-b5e1-4759-b085-77ec08210ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=69fded38b1c44c33827732495be0447b)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:26.135669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b14cb-b5e1-4759-b085-77ec08210ffe admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:26.187759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 963/1950] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:26.188786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df05817-a4ff-4646-aff8-1083598dfeb9 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:26.195455 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5df05817-a4ff-4646-aff8-1083598dfeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:26.215473 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df05817-a4ff-4646-aff8-1083598dfeb9 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:26.232617 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74febe33-79fe-452a-9849-a7b95a17e76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:26.234170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74febe33-79fe-452a-9849-a7b95a17e76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:26.234670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74febe33-79fe-452a-9849-a7b95a17e76c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:26.235455 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74febe33-79fe-452a-9849-a7b95a17e76c admin admin] PATH_INFO: `/v3/users/e09e2cd28e9a4fb99210416e1333903e` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:26.237030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74febe33-79fe-452a-9849-a7b95a17e76c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e09e2cd28e9a4fb99210416e1333903e)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:26.239831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74febe33-79fe-452a-9849-a7b95a17e76c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:26.243562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-168b14cb-b5e1-4759-b085-77ec08210ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69fded38b1c44c33827732495be0447b'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.244321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-168b14cb-b5e1-4759-b085-77ec08210ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69fded38b1c44c33827732495be0447b'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.257837 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:26.284078 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-168b14cb-b5e1-4759-b085-77ec08210ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69fded38b1c44c33827732495be0447b'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.296294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-168b14cb-b5e1-4759-b085-77ec08210ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69fded38b1c44c33827732495be0447b'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.328874 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-168b14cb-b5e1-4759-b085-77ec08210ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69fded38b1c44c33827732495be0447b'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.348134 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 988/1951] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:26 2022] DELETE /identity/v3/users/69fded38b1c44c33827732495be0447b => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:53:26.349923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac08aa-924b-47c7-98d6-23761819bd33 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:26.350267 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac08aa-924b-47c7-98d6-23761819bd33 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:26.350567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac08aa-924b-47c7-98d6-23761819bd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:26.370711 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-74febe33-79fe-452a-9849-a7b95a17e76c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e09e2cd28e9a4fb99210416e1333903e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.381215 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-74febe33-79fe-452a-9849-a7b95a17e76c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e09e2cd28e9a4fb99210416e1333903e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.382081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-74febe33-79fe-452a-9849-a7b95a17e76c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e09e2cd28e9a4fb99210416e1333903e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.387517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-86ac08aa-924b-47c7-98d6-23761819bd33 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:26.388635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-86ac08aa-924b-47c7-98d6-23761819bd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:26.393656 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-74febe33-79fe-452a-9849-a7b95a17e76c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e09e2cd28e9a4fb99210416e1333903e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.394800 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-74febe33-79fe-452a-9849-a7b95a17e76c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e09e2cd28e9a4fb99210416e1333903e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.425930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:26.438094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 964/1952] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:26 2022] DELETE /identity/v3/users/e09e2cd28e9a4fb99210416e1333903e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:26.439443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea08d7f-a97c-4bc9-a91f-53204d6543f5 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:26.447834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea08d7f-a97c-4bc9-a91f-53204d6543f5 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:26.447834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea08d7f-a97c-4bc9-a91f-53204d6543f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:26.464343 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-2ea08d7f-a97c-4bc9-a91f-53204d6543f5 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:26.465636 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-2ea08d7f-a97c-4bc9-a91f-53204d6543f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:26.487374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 989/1953] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:26.487374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b7a59-9819-45ce-a2ac-16ea6ed54dc0 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:26.487374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b7a59-9819-45ce-a2ac-16ea6ed54dc0 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:26.487374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b7a59-9819-45ce-a2ac-16ea6ed54dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:26.509228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-0f2b7a59-9819-45ce-a2ac-16ea6ed54dc0 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:26.510668 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-0f2b7a59-9819-45ce-a2ac-16ea6ed54dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:26.569833 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 965/1954] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:26.572257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea08d7f-a97c-4bc9-a91f-53204d6543f5 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:26.586983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea08d7f-a97c-4bc9-a91f-53204d6543f5 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:26.591565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50763b-34d1-40eb-a460-3b8f2cd8c65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:26.592422 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50763b-34d1-40eb-a460-3b8f2cd8c65a admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:26.592822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50763b-34d1-40eb-a460-3b8f2cd8c65a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:26.593161 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50763b-34d1-40eb-a460-3b8f2cd8c65a admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:26.593961 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50763b-34d1-40eb-a460-3b8f2cd8c65a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:53:26.594338 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50763b-34d1-40eb-a460-3b8f2cd8c65a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:26.596234 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50763b-34d1-40eb-a460-3b8f2cd8c65a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:26.598632 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 990/1955] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:26.600177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b7a59-9819-45ce-a2ac-16ea6ed54dc0 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:26.608410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-0f2b7a59-9819-45ce-a2ac-16ea6ed54dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:26.610499 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 966/1956] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:53:26.611813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50763b-34d1-40eb-a460-3b8f2cd8c65a admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:26.615326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ec50763b-34d1-40eb-a460-3b8f2cd8c65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:26.629165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50763b-34d1-40eb-a460-3b8f2cd8c65a admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:26.635509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6724d0-0484-4c05-b037-48ad82020663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:26.636592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6724d0-0484-4c05-b037-48ad82020663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:26.636879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6724d0-0484-4c05-b037-48ad82020663 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:26.637155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6724d0-0484-4c05-b037-48ad82020663 admin admin] PATH_INFO: `/v3/users/953ed636d6384f7489c2dec83864910b` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:26.639856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6724d0-0484-4c05-b037-48ad82020663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=953ed636d6384f7489c2dec83864910b)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:26.641895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6724d0-0484-4c05-b037-48ad82020663 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:26.703932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b7a59-9819-45ce-a2ac-16ea6ed54dc0 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:26.709000 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ee6724d0-0484-4c05-b037-48ad82020663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '953ed636d6384f7489c2dec83864910b'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.709450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f51ba-5754-4d54-afe5-8668945ebe0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:26.710129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f51ba-5754-4d54-afe5-8668945ebe0d service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:26.710414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f51ba-5754-4d54-afe5-8668945ebe0d service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:26.710688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f51ba-5754-4d54-afe5-8668945ebe0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:26.722531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f51ba-5754-4d54-afe5-8668945ebe0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:26.723486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ee6724d0-0484-4c05-b037-48ad82020663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '953ed636d6384f7489c2dec83864910b'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.724004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f51ba-5754-4d54-afe5-8668945ebe0d service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:26.724736 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ee6724d0-0484-4c05-b037-48ad82020663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '953ed636d6384f7489c2dec83864910b'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.735328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 991/1957] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:26.736591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f51ba-5754-4d54-afe5-8668945ebe0d service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:26.737922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ee6724d0-0484-4c05-b037-48ad82020663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '953ed636d6384f7489c2dec83864910b'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.738040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-017f51ba-5754-4d54-afe5-8668945ebe0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:26.739055 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ee6724d0-0484-4c05-b037-48ad82020663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '953ed636d6384f7489c2dec83864910b'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:26.752334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f51ba-5754-4d54-afe5-8668945ebe0d service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:26.759619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a02002-9b6a-45e0-b0a9-8b6c9e05ab14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:26.761423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a02002-9b6a-45e0-b0a9-8b6c9e05ab14 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:26.761862 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a02002-9b6a-45e0-b0a9-8b6c9e05ab14 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:26.762297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a02002-9b6a-45e0-b0a9-8b6c9e05ab14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:26.768377 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d3a02002-9b6a-45e0-b0a9-8b6c9e05ab14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:26.782648 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 967/1958] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:26 2022] DELETE /identity/v3/users/953ed636d6384f7489c2dec83864910b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:26.784371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6724d0-0484-4c05-b037-48ad82020663 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:26.786954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ee6724d0-0484-4c05-b037-48ad82020663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:26.790162 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a02002-9b6a-45e0-b0a9-8b6c9e05ab14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:26.794721 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a02002-9b6a-45e0-b0a9-8b6c9e05ab14 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:26.801151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:26.816259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 992/1959] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:26.819113 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73060939-514a-4cc8-a97d-47157c58486a None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:26.819418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73060939-514a-4cc8-a97d-47157c58486a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:26.819971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73060939-514a-4cc8-a97d-47157c58486a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:26.861522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-73060939-514a-4cc8-a97d-47157c58486a None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:26.863415 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-73060939-514a-4cc8-a97d-47157c58486a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:26.918957 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:26.927852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6724d0-0484-4c05-b037-48ad82020663 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:26.935398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c980e-0924-4819-888a-71388a406f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:26.936138 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c980e-0924-4819-888a-71388a406f34 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:26.936487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c980e-0924-4819-888a-71388a406f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:26.936778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c980e-0924-4819-888a-71388a406f34 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:26.937521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c980e-0924-4819-888a-71388a406f34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:26.940065 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c980e-0924-4819-888a-71388a406f34 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:26.965075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 993/1960] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:26.976418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 968/1961] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:26 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:26.981960 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73060939-514a-4cc8-a97d-47157c58486a None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:26.992466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c980e-0924-4819-888a-71388a406f34 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:26.995870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e90c980e-0924-4819-888a-71388a406f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:26.997030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73060939-514a-4cc8-a97d-47157c58486a None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:27.099409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cfd22-c9ce-43b8-9fe2-0a2eb22ea4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:27.100521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c980e-0924-4819-888a-71388a406f34 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:27.100797 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cfd22-c9ce-43b8-9fe2-0a2eb22ea4ac admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:27.101242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cfd22-c9ce-43b8-9fe2-0a2eb22ea4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:27.101512 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cfd22-c9ce-43b8-9fe2-0a2eb22ea4ac admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:27.102978 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69cfd22-c9ce-43b8-9fe2-0a2eb22ea4ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:27.105097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69cfd22-c9ce-43b8-9fe2-0a2eb22ea4ac admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:27.105638 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9b58b-991c-47c1-b5ae-5b1ee4f8b7a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:27.106311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9b58b-991c-47c1-b5ae-5b1ee4f8b7a2 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:27.106595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9b58b-991c-47c1-b5ae-5b1ee4f8b7a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:27.106878 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9b58b-991c-47c1-b5ae-5b1ee4f8b7a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:27.117606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9b58b-991c-47c1-b5ae-5b1ee4f8b7a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:27.120078 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9b58b-991c-47c1-b5ae-5b1ee4f8b7a2 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:27.132071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 969/1962] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:27.136144 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 994/1963] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:26 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:53:27.143598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9b58b-991c-47c1-b5ae-5b1ee4f8b7a2 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:27.157626 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9b58b-991c-47c1-b5ae-5b1ee4f8b7a2 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:27.164283 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff2274-d3f0-4865-812b-edd5b1427089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:27.165110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff2274-d3f0-4865-812b-edd5b1427089 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:27.165400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff2274-d3f0-4865-812b-edd5b1427089 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:27.165910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff2274-d3f0-4865-812b-edd5b1427089 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:27.167139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff2274-d3f0-4865-812b-edd5b1427089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:27.168533 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff2274-d3f0-4865-812b-edd5b1427089 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:27.183199 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 970/1964] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:27.187601 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cfd22-c9ce-43b8-9fe2-0a2eb22ea4ac admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:27.202648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cfd22-c9ce-43b8-9fe2-0a2eb22ea4ac admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:27.212616 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93862cfa-200b-4c6a-ae4f-8724e901b622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:27.213890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93862cfa-200b-4c6a-ae4f-8724e901b622 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:27.214291 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93862cfa-200b-4c6a-ae4f-8724e901b622 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:27.218073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93862cfa-200b-4c6a-ae4f-8724e901b622 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:27.218837 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff2274-d3f0-4865-812b-edd5b1427089 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:27.219498 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93862cfa-200b-4c6a-ae4f-8724e901b622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:27.221286 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93862cfa-200b-4c6a-ae4f-8724e901b622 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:27.221949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c7ff2274-d3f0-4865-812b-edd5b1427089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:27.247961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 995/1965] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:27.247961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93862cfa-200b-4c6a-ae4f-8724e901b622 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:27.248915 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff2274-d3f0-4865-812b-edd5b1427089 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:27.254992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8bc7f-1204-46f3-ba8f-5138cad439d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:27.256013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-93862cfa-200b-4c6a-ae4f-8724e901b622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:27.256851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8bc7f-1204-46f3-ba8f-5138cad439d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:27.257150 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8bc7f-1204-46f3-ba8f-5138cad439d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:27.257585 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8bc7f-1204-46f3-ba8f-5138cad439d9 admin admin] PATH_INFO: `/v3/projects/0be25e28ff934eadbaec30761c443d69` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:27.260149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8bc7f-1204-46f3-ba8f-5138cad439d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0be25e28ff934eadbaec30761c443d69)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:27.265880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8bc7f-1204-46f3-ba8f-5138cad439d9 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:27.275792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93862cfa-200b-4c6a-ae4f-8724e901b622 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:27.282996 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:27.284206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:27.284735 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:27.285326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf admin admin] PATH_INFO: `/v3/projects/dc851bc7d9194d50b83133e473992b4d` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:27.286875 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc851bc7d9194d50b83133e473992b4d)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:27.289151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:27.335888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-34b8bc7f-1204-46f3-ba8f-5138cad439d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0be25e28ff934eadbaec30761c443d69', 'request_id': 'req-34b8bc7f-1204-46f3-ba8f-5138cad439d9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:27.345850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-34b8bc7f-1204-46f3-ba8f-5138cad439d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0be25e28ff934eadbaec30761c443d69', 'request_id': 'req-34b8bc7f-1204-46f3-ba8f-5138cad439d9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:27.352895 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc851bc7d9194d50b83133e473992b4d', 'request_id': 'req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:27.354457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-34b8bc7f-1204-46f3-ba8f-5138cad439d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0be25e28ff934eadbaec30761c443d69', 'request_id': 'req-34b8bc7f-1204-46f3-ba8f-5138cad439d9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:27.363384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc851bc7d9194d50b83133e473992b4d', 'request_id': 'req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:27.373598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc851bc7d9194d50b83133e473992b4d', 'request_id': 'req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:27.396512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-34b8bc7f-1204-46f3-ba8f-5138cad439d9 admin admin] The token cache is being invalidate because project 0be25e28ff934eadbaec30761c443d69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:27.396956 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-34b8bc7f-1204-46f3-ba8f-5138cad439d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:27.397818 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-34b8bc7f-1204-46f3-ba8f-5138cad439d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:27.398524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf admin admin] The token cache is being invalidate because project dc851bc7d9194d50b83133e473992b4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:27.399810 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:27.401183 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 971/1966] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:27 2022] DELETE /identity/v3/projects/0be25e28ff934eadbaec30761c443d69 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:53:27.402286 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8bc7f-1204-46f3-ba8f-5138cad439d9 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:27.402805 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:27.404535 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-34b8bc7f-1204-46f3-ba8f-5138cad439d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:27.412959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 996/1967] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:27 2022] DELETE /identity/v3/projects/dc851bc7d9194d50b83133e473992b4d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:27.415951 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:27.418234 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:27.428153 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:27.516093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:27.525260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8bc7f-1204-46f3-ba8f-5138cad439d9 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:27.554520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdb3e3-ceb7-4ef2-83bf-6841415b67cf admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:27.561821 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143025bc-0bc8-41ae-88b8-7bbb5fa34c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:27.562871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143025bc-0bc8-41ae-88b8-7bbb5fa34c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:27.563659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143025bc-0bc8-41ae-88b8-7bbb5fa34c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:27.563659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143025bc-0bc8-41ae-88b8-7bbb5fa34c7f admin admin] PATH_INFO: `/v3/users/3617708a44b44744bc49567d0fd97ad4` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:27.565697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143025bc-0bc8-41ae-88b8-7bbb5fa34c7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3617708a44b44744bc49567d0fd97ad4)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:27.567802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143025bc-0bc8-41ae-88b8-7bbb5fa34c7f admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:27.641702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9292ddb-4f1c-4215-9289-3c2fc4180701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:27.644923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9292ddb-4f1c-4215-9289-3c2fc4180701 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:27.645547 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9292ddb-4f1c-4215-9289-3c2fc4180701 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:27.645855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9292ddb-4f1c-4215-9289-3c2fc4180701 admin admin] PATH_INFO: `/v3/projects/8877aa18458245259526a2d457a6040d/users/df4d94dccf564514b2d450d11775466e/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:27.665598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9292ddb-4f1c-4215-9289-3c2fc4180701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8877aa18458245259526a2d457a6040d, user_id=df4d94dccf564514b2d450d11775466e, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:27.668825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9292ddb-4f1c-4215-9289-3c2fc4180701 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:27.675323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-143025bc-0bc8-41ae-88b8-7bbb5fa34c7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3617708a44b44744bc49567d0fd97ad4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:27.676650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-143025bc-0bc8-41ae-88b8-7bbb5fa34c7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3617708a44b44744bc49567d0fd97ad4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:27.690382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 972/1968] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:27 2022] PUT /identity/v3/projects/8877aa18458245259526a2d457a6040d/users/df4d94dccf564514b2d450d11775466e/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:27.692725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9292ddb-4f1c-4215-9289-3c2fc4180701 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:27.694565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f9292ddb-4f1c-4215-9289-3c2fc4180701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:27.703301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-143025bc-0bc8-41ae-88b8-7bbb5fa34c7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3617708a44b44744bc49567d0fd97ad4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:27.716191 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-143025bc-0bc8-41ae-88b8-7bbb5fa34c7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3617708a44b44744bc49567d0fd97ad4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:27.725341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-143025bc-0bc8-41ae-88b8-7bbb5fa34c7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3617708a44b44744bc49567d0fd97ad4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:27.735209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 997/1969] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:27 2022] DELETE /identity/v3/users/3617708a44b44744bc49567d0fd97ad4 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:27.736946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dbbf9-2aaf-4763-841b-dea020de751e None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:27.737348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dbbf9-2aaf-4763-841b-dea020de751e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:27.737738 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dbbf9-2aaf-4763-841b-dea020de751e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:27.776025 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-c01dbbf9-2aaf-4763-841b-dea020de751e None None] MFA Rules not processed for user `df4d94dccf564514b2d450d11775466e`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:27.778234 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c01dbbf9-2aaf-4763-841b-dea020de751e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:27.801247 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9292ddb-4f1c-4215-9289-3c2fc4180701 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:27.916257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 998/1970] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:27.917306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dbbf9-2aaf-4763-841b-dea020de751e None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:27.918799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c01dbbf9-2aaf-4763-841b-dea020de751e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:27.925591 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f992fe-4850-448d-8989-4b103dcb0d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:27.926550 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f992fe-4850-448d-8989-4b103dcb0d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:27.926837 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f992fe-4850-448d-8989-4b103dcb0d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:27.927766 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f992fe-4850-448d-8989-4b103dcb0d80 admin admin] PATH_INFO: `/v3/users/13464f36946e4e1a8f6ff2b665f9bba7` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:27.929070 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f992fe-4850-448d-8989-4b103dcb0d80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13464f36946e4e1a8f6ff2b665f9bba7)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:27.931889 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f992fe-4850-448d-8989-4b103dcb0d80 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:27.937525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dbbf9-2aaf-4763-841b-dea020de751e None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:27.942809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80f1b7-4580-4b8f-9553-6af814bb76ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:27.943696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80f1b7-4580-4b8f-9553-6af814bb76ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:27.944007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80f1b7-4580-4b8f-9553-6af814bb76ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:27.944287 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80f1b7-4580-4b8f-9553-6af814bb76ad admin admin] PATH_INFO: `/v3/projects/822e004d7a4245e0bba093ce0f7a3256` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:27.945667 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80f1b7-4580-4b8f-9553-6af814bb76ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=822e004d7a4245e0bba093ce0f7a3256)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:27.948035 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80f1b7-4580-4b8f-9553-6af814bb76ad admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:27.957490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:27.985047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:28.012493 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b9f992fe-4850-448d-8989-4b103dcb0d80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13464f36946e4e1a8f6ff2b665f9bba7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.023085 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b9f992fe-4850-448d-8989-4b103dcb0d80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13464f36946e4e1a8f6ff2b665f9bba7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.023298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9b80f1b7-4580-4b8f-9553-6af814bb76ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '822e004d7a4245e0bba093ce0f7a3256', 'request_id': 'req-9b80f1b7-4580-4b8f-9553-6af814bb76ad'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.024867 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b9f992fe-4850-448d-8989-4b103dcb0d80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13464f36946e4e1a8f6ff2b665f9bba7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.032286 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9b80f1b7-4580-4b8f-9553-6af814bb76ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '822e004d7a4245e0bba093ce0f7a3256', 'request_id': 'req-9b80f1b7-4580-4b8f-9553-6af814bb76ad'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.035770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b9f992fe-4850-448d-8989-4b103dcb0d80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13464f36946e4e1a8f6ff2b665f9bba7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.036642 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b9f992fe-4850-448d-8989-4b103dcb0d80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13464f36946e4e1a8f6ff2b665f9bba7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.040122 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9b80f1b7-4580-4b8f-9553-6af814bb76ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '822e004d7a4245e0bba093ce0f7a3256', 'request_id': 'req-9b80f1b7-4580-4b8f-9553-6af814bb76ad'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.057310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9b80f1b7-4580-4b8f-9553-6af814bb76ad admin admin] The token cache is being invalidate because project 822e004d7a4245e0bba093ce0f7a3256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:28.057310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9b80f1b7-4580-4b8f-9553-6af814bb76ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.058806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9b80f1b7-4580-4b8f-9553-6af814bb76ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.060771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 999/1971] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:27 2022] DELETE /identity/v3/projects/822e004d7a4245e0bba093ce0f7a3256 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:28.062457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80f1b7-4580-4b8f-9553-6af814bb76ad admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:28.063456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9b80f1b7-4580-4b8f-9553-6af814bb76ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:28.075543 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 973/1972] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:27 2022] DELETE /identity/v3/users/13464f36946e4e1a8f6ff2b665f9bba7 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:28.080578 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f992fe-4850-448d-8989-4b103dcb0d80 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:28.082695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b9f992fe-4850-448d-8989-4b103dcb0d80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:28.197798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80f1b7-4580-4b8f-9553-6af814bb76ad admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:28.214463 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f992fe-4850-448d-8989-4b103dcb0d80 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:28.230439 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab9285-8acc-4c0f-a1af-4ce21bb28624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:28.232065 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab9285-8acc-4c0f-a1af-4ce21bb28624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:28.232564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab9285-8acc-4c0f-a1af-4ce21bb28624 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:28.235212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab9285-8acc-4c0f-a1af-4ce21bb28624 admin admin] PATH_INFO: `/v3/users/f937daeb83e94358ac1bc9362ccc5ba8` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:28.238074 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab9285-8acc-4c0f-a1af-4ce21bb28624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f937daeb83e94358ac1bc9362ccc5ba8)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:28.241308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab9285-8acc-4c0f-a1af-4ce21bb28624 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:28.291518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdaa2ff-e545-4ffc-953a-ad63e0be958f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:28.292660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdaa2ff-e545-4ffc-953a-ad63e0be958f service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:28.293132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdaa2ff-e545-4ffc-953a-ad63e0be958f service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:28.293903 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdaa2ff-e545-4ffc-953a-ad63e0be958f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:28.297565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-abdaa2ff-e545-4ffc-953a-ad63e0be958f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:28.306347 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a4ab9285-8acc-4c0f-a1af-4ce21bb28624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f937daeb83e94358ac1bc9362ccc5ba8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.324108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a4ab9285-8acc-4c0f-a1af-4ce21bb28624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f937daeb83e94358ac1bc9362ccc5ba8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.326156 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a4ab9285-8acc-4c0f-a1af-4ce21bb28624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f937daeb83e94358ac1bc9362ccc5ba8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.348997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a4ab9285-8acc-4c0f-a1af-4ce21bb28624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f937daeb83e94358ac1bc9362ccc5ba8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.350259 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a4ab9285-8acc-4c0f-a1af-4ce21bb28624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f937daeb83e94358ac1bc9362ccc5ba8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.379861 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 974/1973] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:28 2022] DELETE /identity/v3/users/f937daeb83e94358ac1bc9362ccc5ba8 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:28.412209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdaa2ff-e545-4ffc-953a-ad63e0be958f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:28.421405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdaa2ff-e545-4ffc-953a-ad63e0be958f service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:28.449457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1000/1974] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:28.476254 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab9285-8acc-4c0f-a1af-4ce21bb28624 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:28.479488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a4ab9285-8acc-4c0f-a1af-4ce21bb28624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:28.563099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdaa2ff-e545-4ffc-953a-ad63e0be958f service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:28.565033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-abdaa2ff-e545-4ffc-953a-ad63e0be958f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:28.578969 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:28.593500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdaa2ff-e545-4ffc-953a-ad63e0be958f service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:28.604737 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:28.696565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab9285-8acc-4c0f-a1af-4ce21bb28624 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:28.711872 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69bdae-46b9-423a-a423-0a9f82bfe695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:28.712710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69bdae-46b9-423a-a423-0a9f82bfe695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:28.713074 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69bdae-46b9-423a-a423-0a9f82bfe695 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:28.721124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69bdae-46b9-423a-a423-0a9f82bfe695 admin admin] PATH_INFO: `/v3/projects/e75bd814be1148dfb85d2103a2e4f6ab` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:28.721124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69bdae-46b9-423a-a423-0a9f82bfe695 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e75bd814be1148dfb85d2103a2e4f6ab)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:28.722774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69bdae-46b9-423a-a423-0a9f82bfe695 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:28.741870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293d6d5-321a-40a7-bedb-a785da0ee6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:28.743942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293d6d5-321a-40a7-bedb-a785da0ee6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:28.744569 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293d6d5-321a-40a7-bedb-a785da0ee6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:28.744988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293d6d5-321a-40a7-bedb-a785da0ee6f0 admin admin] PATH_INFO: `/v3/projects/f3d740ba7e7443e386aa9e1d6285c55d` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:28.746567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293d6d5-321a-40a7-bedb-a785da0ee6f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3d740ba7e7443e386aa9e1d6285c55d)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:28.748785 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293d6d5-321a-40a7-bedb-a785da0ee6f0 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:28.793347 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-fa69bdae-46b9-423a-a423-0a9f82bfe695 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e75bd814be1148dfb85d2103a2e4f6ab', 'request_id': 'req-fa69bdae-46b9-423a-a423-0a9f82bfe695'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.803631 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-fa69bdae-46b9-423a-a423-0a9f82bfe695 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e75bd814be1148dfb85d2103a2e4f6ab', 'request_id': 'req-fa69bdae-46b9-423a-a423-0a9f82bfe695'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.813710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-fa69bdae-46b9-423a-a423-0a9f82bfe695 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e75bd814be1148dfb85d2103a2e4f6ab', 'request_id': 'req-fa69bdae-46b9-423a-a423-0a9f82bfe695'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.828883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7293d6d5-321a-40a7-bedb-a785da0ee6f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3d740ba7e7443e386aa9e1d6285c55d', 'request_id': 'req-7293d6d5-321a-40a7-bedb-a785da0ee6f0'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.835501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-fa69bdae-46b9-423a-a423-0a9f82bfe695 admin admin] The token cache is being invalidate because project e75bd814be1148dfb85d2103a2e4f6ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:28.835888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-fa69bdae-46b9-423a-a423-0a9f82bfe695 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.837671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-fa69bdae-46b9-423a-a423-0a9f82bfe695 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.838394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7293d6d5-321a-40a7-bedb-a785da0ee6f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3d740ba7e7443e386aa9e1d6285c55d', 'request_id': 'req-7293d6d5-321a-40a7-bedb-a785da0ee6f0'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.845436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7293d6d5-321a-40a7-bedb-a785da0ee6f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3d740ba7e7443e386aa9e1d6285c55d', 'request_id': 'req-7293d6d5-321a-40a7-bedb-a785da0ee6f0'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.847911 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 975/1975] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:28 2022] DELETE /identity/v3/projects/e75bd814be1148dfb85d2103a2e4f6ab => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:28.850544 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69bdae-46b9-423a-a423-0a9f82bfe695 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:28.861324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-fa69bdae-46b9-423a-a423-0a9f82bfe695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:28.872177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7293d6d5-321a-40a7-bedb-a785da0ee6f0 admin admin] The token cache is being invalidate because project f3d740ba7e7443e386aa9e1d6285c55d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:28.872882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7293d6d5-321a-40a7-bedb-a785da0ee6f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.873950 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7293d6d5-321a-40a7-bedb-a785da0ee6f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:28.877419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1001/1976] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:28 2022] DELETE /identity/v3/projects/f3d740ba7e7443e386aa9e1d6285c55d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:28.884065 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7de02-5344-49ba-8f80-50ba102c9e67 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:28.885185 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7de02-5344-49ba-8f80-50ba102c9e67 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:28.885799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7de02-5344-49ba-8f80-50ba102c9e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:28.913036 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-f5d7de02-5344-49ba-8f80-50ba102c9e67 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:28.916843 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f5d7de02-5344-49ba-8f80-50ba102c9e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:29.010307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69bdae-46b9-423a-a423-0a9f82bfe695 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:29.021682 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1002/1977] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:29.030964 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af613084-3edc-4c52-a9f4-41c9133f9a6a None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.031356 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af613084-3edc-4c52-a9f4-41c9133f9a6a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.031788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af613084-3edc-4c52-a9f4-41c9133f9a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.069256 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-af613084-3edc-4c52-a9f4-41c9133f9a6a None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:29.070788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-af613084-3edc-4c52-a9f4-41c9133f9a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:29.082850 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1003/1978] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:29.087680 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af613084-3edc-4c52-a9f4-41c9133f9a6a None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:29.103786 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af613084-3edc-4c52-a9f4-41c9133f9a6a None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:29.126807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc02f17-16a4-42e0-999f-d51d0f6164a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:29.127997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc02f17-16a4-42e0-999f-d51d0f6164a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.128283 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc02f17-16a4-42e0-999f-d51d0f6164a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.131283 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc02f17-16a4-42e0-999f-d51d0f6164a9 admin admin] PATH_INFO: `/v3/users/34f20989c7434905b4eb87d80fec8620` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.131283 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc02f17-16a4-42e0-999f-d51d0f6164a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34f20989c7434905b4eb87d80fec8620)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:29.131596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e44df4-c136-4dd5-9792-71d72d3453ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:29.132306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e44df4-c136-4dd5-9792-71d72d3453ac admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.132597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e44df4-c136-4dd5-9792-71d72d3453ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.132765 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc02f17-16a4-42e0-999f-d51d0f6164a9 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:29.132880 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e44df4-c136-4dd5-9792-71d72d3453ac admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.134219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e44df4-c136-4dd5-9792-71d72d3453ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:53:29.134219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e44df4-c136-4dd5-9792-71d72d3453ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:29.135828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e44df4-c136-4dd5-9792-71d72d3453ac admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:29.148182 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:29.149458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1004/1979] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:53:29.150453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e44df4-c136-4dd5-9792-71d72d3453ac admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:29.156470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-06e44df4-c136-4dd5-9792-71d72d3453ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:29.199100 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e44df4-c136-4dd5-9792-71d72d3453ac admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:29.207799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44507982-b6bc-4e17-b346-499818e8b839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:29.208590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0fc02f17-16a4-42e0-999f-d51d0f6164a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34f20989c7434905b4eb87d80fec8620'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:29.209876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44507982-b6bc-4e17-b346-499818e8b839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.210632 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44507982-b6bc-4e17-b346-499818e8b839 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.211395 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44507982-b6bc-4e17-b346-499818e8b839 admin admin] PATH_INFO: `/v3/projects/1213b68a36094fd9849498b886cc242f` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.214094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44507982-b6bc-4e17-b346-499818e8b839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1213b68a36094fd9849498b886cc242f)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:29.218360 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44507982-b6bc-4e17-b346-499818e8b839 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:29.225314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0fc02f17-16a4-42e0-999f-d51d0f6164a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34f20989c7434905b4eb87d80fec8620'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:29.226166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0fc02f17-16a4-42e0-999f-d51d0f6164a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34f20989c7434905b4eb87d80fec8620'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:29.236853 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:29.239091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0fc02f17-16a4-42e0-999f-d51d0f6164a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34f20989c7434905b4eb87d80fec8620'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:29.240723 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0fc02f17-16a4-42e0-999f-d51d0f6164a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34f20989c7434905b4eb87d80fec8620'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:29.275786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 976/1980] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:28 2022] DELETE /identity/v3/users/34f20989c7434905b4eb87d80fec8620 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:29.275786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc02f17-16a4-42e0-999f-d51d0f6164a9 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:29.275786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-0fc02f17-16a4-42e0-999f-d51d0f6164a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:29.302499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-44507982-b6bc-4e17-b346-499818e8b839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1213b68a36094fd9849498b886cc242f', 'request_id': 'req-44507982-b6bc-4e17-b346-499818e8b839'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:29.317380 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-44507982-b6bc-4e17-b346-499818e8b839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1213b68a36094fd9849498b886cc242f', 'request_id': 'req-44507982-b6bc-4e17-b346-499818e8b839'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:29.326897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-44507982-b6bc-4e17-b346-499818e8b839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1213b68a36094fd9849498b886cc242f', 'request_id': 'req-44507982-b6bc-4e17-b346-499818e8b839'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:29.353784 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-44507982-b6bc-4e17-b346-499818e8b839 admin admin] The token cache is being invalidate because project 1213b68a36094fd9849498b886cc242f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:29.354773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-44507982-b6bc-4e17-b346-499818e8b839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:29.355700 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-44507982-b6bc-4e17-b346-499818e8b839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:29.359208 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1005/1981] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:29 2022] DELETE /identity/v3/projects/1213b68a36094fd9849498b886cc242f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:29.364779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59280dd5-ccb7-457e-a02b-e8bcda39f767 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.369544 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59280dd5-ccb7-457e-a02b-e8bcda39f767 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.370331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59280dd5-ccb7-457e-a02b-e8bcda39f767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.378649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc02f17-16a4-42e0-999f-d51d0f6164a9 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:29.409858 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-59280dd5-ccb7-457e-a02b-e8bcda39f767 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:29.411099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-59280dd5-ccb7-457e-a02b-e8bcda39f767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:29.463781 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb4389-9ba7-4680-814b-28c3bb586b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:29.463781 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb4389-9ba7-4680-814b-28c3bb586b16 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.463781 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb4389-9ba7-4680-814b-28c3bb586b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.463781 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb4389-9ba7-4680-814b-28c3bb586b16 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.464420 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb4389-9ba7-4680-814b-28c3bb586b16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:29.464728 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb4389-9ba7-4680-814b-28c3bb586b16 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:29.503359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 977/1982] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:29 2022] POST /identity/v3/projects => generated 387 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:29.505105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb4389-9ba7-4680-814b-28c3bb586b16 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:29.507870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-cafb4389-9ba7-4680-814b-28c3bb586b16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:29.511920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1006/1983] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:29.518240 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773cf87-4950-443b-bf94-ac480d31de34 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.518240 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773cf87-4950-443b-bf94-ac480d31de34 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.518240 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773cf87-4950-443b-bf94-ac480d31de34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.539231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e773cf87-4950-443b-bf94-ac480d31de34 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:29.540488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:29.541575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e773cf87-4950-443b-bf94-ac480d31de34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:29.598612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb4389-9ba7-4680-814b-28c3bb586b16 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:29.604386 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c827e-caca-49ad-a786-ecfb3ab4e56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:29.605224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c827e-caca-49ad-a786-ecfb3ab4e56a admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.605514 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c827e-caca-49ad-a786-ecfb3ab4e56a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.605964 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c827e-caca-49ad-a786-ecfb3ab4e56a admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.606687 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c827e-caca-49ad-a786-ecfb3ab4e56a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:29.608694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c827e-caca-49ad-a786-ecfb3ab4e56a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:29.619380 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1007/1984] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:29.620907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773cf87-4950-443b-bf94-ac480d31de34 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:29.632744 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773cf87-4950-443b-bf94-ac480d31de34 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:29.642711 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 978/1985] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:29 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:29.647113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c827e-caca-49ad-a786-ecfb3ab4e56a admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:29.649633 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5eb2e-3ce5-409d-a400-a600acb3b4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:29.650388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5eb2e-3ce5-409d-a400-a600acb3b4af admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.650733 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5eb2e-3ce5-409d-a400-a600acb3b4af admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.651075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5eb2e-3ce5-409d-a400-a600acb3b4af admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.651918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5eb2e-3ce5-409d-a400-a600acb3b4af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:53:29.652354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5eb2e-3ce5-409d-a400-a600acb3b4af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:29.656477 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5eb2e-3ce5-409d-a400-a600acb3b4af admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:29.659145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c827e-caca-49ad-a786-ecfb3ab4e56a admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:29.665120 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00063113-b922-43fb-a357-8acecdc9ba28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:29.666764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00063113-b922-43fb-a357-8acecdc9ba28 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.667086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00063113-b922-43fb-a357-8acecdc9ba28 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.667443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00063113-b922-43fb-a357-8acecdc9ba28 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.668299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00063113-b922-43fb-a357-8acecdc9ba28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:29.670193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00063113-b922-43fb-a357-8acecdc9ba28 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:29.675178 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1008/1986] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:53:29.676184 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5eb2e-3ce5-409d-a400-a600acb3b4af admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:29.681319 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-56b5eb2e-3ce5-409d-a400-a600acb3b4af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:29.681931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 979/1987] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:53:29.682643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00063113-b922-43fb-a357-8acecdc9ba28 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:29.693043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00063113-b922-43fb-a357-8acecdc9ba28 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:29.695660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5eb2e-3ce5-409d-a400-a600acb3b4af admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:29.700133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c876b07-6539-4eb0-b19a-6dd2a66e0a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:29.700889 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c876b07-6539-4eb0-b19a-6dd2a66e0a5e admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.701190 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cc226-908d-43a3-a992-86dceb0b1fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:29.701322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c876b07-6539-4eb0-b19a-6dd2a66e0a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.701654 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c876b07-6539-4eb0-b19a-6dd2a66e0a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.702053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cc226-908d-43a3-a992-86dceb0b1fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.702336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cc226-908d-43a3-a992-86dceb0b1fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.702581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c876b07-6539-4eb0-b19a-6dd2a66e0a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:29.702747 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cc226-908d-43a3-a992-86dceb0b1fcb admin admin] PATH_INFO: `/v3/projects/b045112b7c4d47bb8735b8a352efda81` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.703909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202cc226-908d-43a3-a992-86dceb0b1fcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b045112b7c4d47bb8735b8a352efda81)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:29.704767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c876b07-6539-4eb0-b19a-6dd2a66e0a5e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:29.705858 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202cc226-908d-43a3-a992-86dceb0b1fcb admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:29.733267 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:29.747802 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 980/1988] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:29 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:29.747802 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c876b07-6539-4eb0-b19a-6dd2a66e0a5e admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:29.769649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-202cc226-908d-43a3-a992-86dceb0b1fcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b045112b7c4d47bb8735b8a352efda81', 'request_id': 'req-202cc226-908d-43a3-a992-86dceb0b1fcb'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:29.780325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-202cc226-908d-43a3-a992-86dceb0b1fcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b045112b7c4d47bb8735b8a352efda81', 'request_id': 'req-202cc226-908d-43a3-a992-86dceb0b1fcb'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:29.787797 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-202cc226-908d-43a3-a992-86dceb0b1fcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b045112b7c4d47bb8735b8a352efda81', 'request_id': 'req-202cc226-908d-43a3-a992-86dceb0b1fcb'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:29.807482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-202cc226-908d-43a3-a992-86dceb0b1fcb admin admin] The token cache is being invalidate because project b045112b7c4d47bb8735b8a352efda81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:29.808130 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-202cc226-908d-43a3-a992-86dceb0b1fcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:29.809067 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-202cc226-908d-43a3-a992-86dceb0b1fcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:29.810733 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1009/1989] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:29 2022] DELETE /identity/v3/projects/b045112b7c4d47bb8735b8a352efda81 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:29.811492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cc226-908d-43a3-a992-86dceb0b1fcb admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:29.813490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-202cc226-908d-43a3-a992-86dceb0b1fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:29.821001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c876b07-6539-4eb0-b19a-6dd2a66e0a5e admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:29.889308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cc226-908d-43a3-a992-86dceb0b1fcb admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:29.890695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdb26e-5ce2-4c59-96c6-a454108bfa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:29.891380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdb26e-5ce2-4c59-96c6-a454108bfa19 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.891667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdb26e-5ce2-4c59-96c6-a454108bfa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.891976 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdb26e-5ce2-4c59-96c6-a454108bfa19 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.892773 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecdb26e-5ce2-4c59-96c6-a454108bfa19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:29.894365 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecdb26e-5ce2-4c59-96c6-a454108bfa19 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:29.894772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ba5d3-a99e-47ef-a3b8-85358d72766a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:29.895439 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ba5d3-a99e-47ef-a3b8-85358d72766a admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.895740 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ba5d3-a99e-47ef-a3b8-85358d72766a admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.896036 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ba5d3-a99e-47ef-a3b8-85358d72766a admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.896737 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ba5d3-a99e-47ef-a3b8-85358d72766a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:29.898698 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ba5d3-a99e-47ef-a3b8-85358d72766a admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:29.905537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 981/1990] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:29.906930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6a90e-f9e1-4f00-a216-683f9f546c41 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.907174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6a90e-f9e1-4f00-a216-683f9f546c41 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.907455 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6a90e-f9e1-4f00-a216-683f9f546c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.930132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1010/1991] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:29 2022] POST /identity/v3/users => generated 283 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:29.930132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ba5d3-a99e-47ef-a3b8-85358d72766a admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:29.931049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-490ba5d3-a99e-47ef-a3b8-85358d72766a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:29.932634 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-e0d6a90e-f9e1-4f00-a216-683f9f546c41 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:29.933948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e0d6a90e-f9e1-4f00-a216-683f9f546c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:29.942521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 982/1992] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:29.943245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ba5d3-a99e-47ef-a3b8-85358d72766a admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:29.943578 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6a90e-f9e1-4f00-a216-683f9f546c41 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:29.948699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edccc93-ae3e-49b4-9313-8be145dd09a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:29.949841 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edccc93-ae3e-49b4-9313-8be145dd09a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.950130 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edccc93-ae3e-49b4-9313-8be145dd09a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.950409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edccc93-ae3e-49b4-9313-8be145dd09a9 admin admin] PATH_INFO: `/v3/projects/e4c00779f3c5422ca29a17f07e97498b/users/55bd0fb684514eabb4cd8345af2def6f/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.955081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6a90e-f9e1-4f00-a216-683f9f546c41 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:29.959924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b8ddc-b08e-4f1d-80ba-0e81bb1a7e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:29.960601 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b8ddc-b08e-4f1d-80ba-0e81bb1a7e70 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.960886 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b8ddc-b08e-4f1d-80ba-0e81bb1a7e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.961167 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b8ddc-b08e-4f1d-80ba-0e81bb1a7e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.962310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b8ddc-b08e-4f1d-80ba-0e81bb1a7e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:29.964117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b8ddc-b08e-4f1d-80ba-0e81bb1a7e70 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:29.970121 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edccc93-ae3e-49b4-9313-8be145dd09a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4c00779f3c5422ca29a17f07e97498b, user_id=55bd0fb684514eabb4cd8345af2def6f, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:29.972929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edccc93-ae3e-49b4-9313-8be145dd09a9 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:29.975645 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 983/1993] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:29.977812 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d06f4-bfcf-4239-b1fc-3b4852a10e73 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:29.978240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d06f4-bfcf-4239-b1fc-3b4852a10e73 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:29.978778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d06f4-bfcf-4239-b1fc-3b4852a10e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:29.986503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:30.002134 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-df5d06f4-bfcf-4239-b1fc-3b4852a10e73 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:30.002461 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1011/1994] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:29 2022] PUT /identity/v3/projects/e4c00779f3c5422ca29a17f07e97498b/users/55bd0fb684514eabb4cd8345af2def6f/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:30.003430 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edccc93-ae3e-49b4-9313-8be145dd09a9 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.003637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-df5d06f4-bfcf-4239-b1fc-3b4852a10e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:30.089909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edccc93-ae3e-49b4-9313-8be145dd09a9 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:30.094881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a002628-9d09-468e-9115-eee3a5515fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:30.095561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a002628-9d09-468e-9115-eee3a5515fdd admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.095663 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 984/1995] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:30.095876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a002628-9d09-468e-9115-eee3a5515fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.096148 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a002628-9d09-468e-9115-eee3a5515fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.096931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4d919-90be-4205-89bb-f689d0ec7c77 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.097021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a002628-9d09-468e-9115-eee3a5515fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:30.097219 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4d919-90be-4205-89bb-f689d0ec7c77 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.097501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4d919-90be-4205-89bb-f689d0ec7c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.098667 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a002628-9d09-468e-9115-eee3a5515fdd admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:30.113599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1012/1996] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:30.113599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a002628-9d09-468e-9115-eee3a5515fdd admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.122383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a002628-9d09-468e-9115-eee3a5515fdd admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:30.127109 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e449fbd-2d0f-49f8-9b8f-2dc17ccb5dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:30.127803 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e449fbd-2d0f-49f8-9b8f-2dc17ccb5dfb admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.128094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e449fbd-2d0f-49f8-9b8f-2dc17ccb5dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.128462 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e449fbd-2d0f-49f8-9b8f-2dc17ccb5dfb admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.129198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e449fbd-2d0f-49f8-9b8f-2dc17ccb5dfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:53:30.129512 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e449fbd-2d0f-49f8-9b8f-2dc17ccb5dfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:30.131129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e449fbd-2d0f-49f8-9b8f-2dc17ccb5dfb admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:30.135769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-6dc4d919-90be-4205-89bb-f689d0ec7c77 None None] MFA Rules not processed for user `55bd0fb684514eabb4cd8345af2def6f`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:30.137144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6dc4d919-90be-4205-89bb-f689d0ec7c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:30.139644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:30.146365 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1013/1997] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:53:30.148209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e449fbd-2d0f-49f8-9b8f-2dc17ccb5dfb admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.160701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e449fbd-2d0f-49f8-9b8f-2dc17ccb5dfb admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:30.168587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76998aa-5bf1-43bb-83ed-967792acf3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:30.169638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76998aa-5bf1-43bb-83ed-967792acf3fd admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.170004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76998aa-5bf1-43bb-83ed-967792acf3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.170391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76998aa-5bf1-43bb-83ed-967792acf3fd admin admin] PATH_INFO: `/v3/projects/c4b5ef2f930f49b2a1653d943bbe96cd/users/ce2f24f2a9404f608017dd019fc08fbe/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.190826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76998aa-5bf1-43bb-83ed-967792acf3fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b5ef2f930f49b2a1653d943bbe96cd, user_id=ce2f24f2a9404f608017dd019fc08fbe, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:30.193826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76998aa-5bf1-43bb-83ed-967792acf3fd admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:30.211966 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1014/1998] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:30 2022] PUT /identity/v3/projects/c4b5ef2f930f49b2a1653d943bbe96cd/users/ce2f24f2a9404f608017dd019fc08fbe/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:30.213087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76998aa-5bf1-43bb-83ed-967792acf3fd admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.220332 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 985/1999] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:30.226996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02262c-3416-4b10-bf26-71d59f08067f None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.227990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02262c-3416-4b10-bf26-71d59f08067f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.228409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02262c-3416-4b10-bf26-71d59f08067f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.277227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-bd02262c-3416-4b10-bf26-71d59f08067f None None] MFA Rules not processed for user `ce2f24f2a9404f608017dd019fc08fbe`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:30.277880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-bd02262c-3416-4b10-bf26-71d59f08067f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:30.316537 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76998aa-5bf1-43bb-83ed-967792acf3fd admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:30.348013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc033a8b-2a81-4733-8d07-67f359da05b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:30.348013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc033a8b-2a81-4733-8d07-67f359da05b6 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.348013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc033a8b-2a81-4733-8d07-67f359da05b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.348013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc033a8b-2a81-4733-8d07-67f359da05b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.348509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc033a8b-2a81-4733-8d07-67f359da05b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:30.348509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc033a8b-2a81-4733-8d07-67f359da05b6 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:30.381629 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 986/2000] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:30.381828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1015/2001] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:30 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:30.381906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02262c-3416-4b10-bf26-71d59f08067f None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.384933 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc033a8b-2a81-4733-8d07-67f359da05b6 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.396244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02262c-3416-4b10-bf26-71d59f08067f None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:30.432665 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:30.492550 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996b87a-65ff-4964-93de-ddd9dc3efc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:30.493624 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996b87a-65ff-4964-93de-ddd9dc3efc6d admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.494072 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996b87a-65ff-4964-93de-ddd9dc3efc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.494352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc033a8b-2a81-4733-8d07-67f359da05b6 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:30.494517 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996b87a-65ff-4964-93de-ddd9dc3efc6d admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.495698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996b87a-65ff-4964-93de-ddd9dc3efc6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:30.499254 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996b87a-65ff-4964-93de-ddd9dc3efc6d admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:30.501475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f498a-44cd-40bc-8b33-d4b0747ca96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:30.502521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f498a-44cd-40bc-8b33-d4b0747ca96e admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.502968 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f498a-44cd-40bc-8b33-d4b0747ca96e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.503964 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f498a-44cd-40bc-8b33-d4b0747ca96e admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.505452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f498a-44cd-40bc-8b33-d4b0747ca96e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:30.507506 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f498a-44cd-40bc-8b33-d4b0747ca96e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:30.537871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1016/2002] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:30 2022] POST /identity/v3/projects => generated 363 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:30.540167 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f498a-44cd-40bc-8b33-d4b0747ca96e admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.546171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 987/2003] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:30 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:30.546535 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996b87a-65ff-4964-93de-ddd9dc3efc6d admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.618674 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:30.636408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996b87a-65ff-4964-93de-ddd9dc3efc6d admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:30.659214 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab504c-c0eb-4fa8-b1ee-8556f713c639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:30.659860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab504c-c0eb-4fa8-b1ee-8556f713c639 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.660216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab504c-c0eb-4fa8-b1ee-8556f713c639 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.660535 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab504c-c0eb-4fa8-b1ee-8556f713c639 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.661443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab504c-c0eb-4fa8-b1ee-8556f713c639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:30.663745 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab504c-c0eb-4fa8-b1ee-8556f713c639 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:30.677886 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f498a-44cd-40bc-8b33-d4b0747ca96e admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:30.686984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b441cc3-c822-4342-8c72-367402d16ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:30.687778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b441cc3-c822-4342-8c72-367402d16ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.688239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b441cc3-c822-4342-8c72-367402d16ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.688635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b441cc3-c822-4342-8c72-367402d16ca7 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.689425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b441cc3-c822-4342-8c72-367402d16ca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:30.691588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b441cc3-c822-4342-8c72-367402d16ca7 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:30.697233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 988/2004] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:30 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:30.698794 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab504c-c0eb-4fa8-b1ee-8556f713c639 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.710565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab504c-c0eb-4fa8-b1ee-8556f713c639 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:30.720076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd04e62-a065-40de-8fb8-67c28cbf801f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:30.720879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd04e62-a065-40de-8fb8-67c28cbf801f admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.721235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd04e62-a065-40de-8fb8-67c28cbf801f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.722715 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd04e62-a065-40de-8fb8-67c28cbf801f admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.724920 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd04e62-a065-40de-8fb8-67c28cbf801f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:30.727035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd04e62-a065-40de-8fb8-67c28cbf801f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:30.728022 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1017/2005] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:30 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:30.728953 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b441cc3-c822-4342-8c72-367402d16ca7 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.740614 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b441cc3-c822-4342-8c72-367402d16ca7 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:30.747175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d510c-c75e-4397-b487-09b0d395fe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:30.748009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d510c-c75e-4397-b487-09b0d395fe37 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.748349 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d510c-c75e-4397-b487-09b0d395fe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.748716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d510c-c75e-4397-b487-09b0d395fe37 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.749664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d510c-c75e-4397-b487-09b0d395fe37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:30.751447 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d510c-c75e-4397-b487-09b0d395fe37 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:30.768821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 989/2006] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:30 2022] POST /identity/v3/users => generated 299 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:30.769967 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd04e62-a065-40de-8fb8-67c28cbf801f admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.771885 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1018/2007] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:30.777298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d510c-c75e-4397-b487-09b0d395fe37 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.782054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd04e62-a065-40de-8fb8-67c28cbf801f admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:30.787066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9188b1f-e4cf-4bdd-b284-f49e44a2b9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:30.787781 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9188b1f-e4cf-4bdd-b284-f49e44a2b9c3 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.788073 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9188b1f-e4cf-4bdd-b284-f49e44a2b9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.788352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9188b1f-e4cf-4bdd-b284-f49e44a2b9c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.789094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9188b1f-e4cf-4bdd-b284-f49e44a2b9c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:30.789635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d510c-c75e-4397-b487-09b0d395fe37 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:30.790767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9188b1f-e4cf-4bdd-b284-f49e44a2b9c3 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:30.795639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b8f6c-3f32-447d-a890-b6fed5771a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:30.797147 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b8f6c-3f32-447d-a890-b6fed5771a8b admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.797432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b8f6c-3f32-447d-a890-b6fed5771a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.797704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b8f6c-3f32-447d-a890-b6fed5771a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.798438 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b8f6c-3f32-447d-a890-b6fed5771a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:30.800084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b8f6c-3f32-447d-a890-b6fed5771a8b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:30.807057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 990/2008] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:30.808833 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9188b1f-e4cf-4bdd-b284-f49e44a2b9c3 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.816290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1019/2009] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:53:30.817012 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b8f6c-3f32-447d-a890-b6fed5771a8b admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.823115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9188b1f-e4cf-4bdd-b284-f49e44a2b9c3 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:30.828431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482fc22-f17c-4a8d-bdeb-61493ef4b5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:30.829347 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482fc22-f17c-4a8d-bdeb-61493ef4b5e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.829664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482fc22-f17c-4a8d-bdeb-61493ef4b5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.829776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b8f6c-3f32-447d-a890-b6fed5771a8b admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:30.829923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482fc22-f17c-4a8d-bdeb-61493ef4b5e2 admin admin] PATH_INFO: `/v3/projects/aea4d2b8b59d46ed8f0c1c0ce42b6341/users/3c09acbd99fe4a1dacdd2c097c8f75d8/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.834681 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef4ccb-8f56-4489-9330-3d582f8c68ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:30.835448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef4ccb-8f56-4489-9330-3d582f8c68ba admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.835820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef4ccb-8f56-4489-9330-3d582f8c68ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.836164 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef4ccb-8f56-4489-9330-3d582f8c68ba admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.837743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef4ccb-8f56-4489-9330-3d582f8c68ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:30.839668 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef4ccb-8f56-4489-9330-3d582f8c68ba admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:30.850777 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1020/2010] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:30.851602 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef4ccb-8f56-4489-9330-3d582f8c68ba admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.854251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482fc22-f17c-4a8d-bdeb-61493ef4b5e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea4d2b8b59d46ed8f0c1c0ce42b6341, user_id=3c09acbd99fe4a1dacdd2c097c8f75d8, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:30.857312 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482fc22-f17c-4a8d-bdeb-61493ef4b5e2 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:30.862252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef4ccb-8f56-4489-9330-3d582f8c68ba admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:30.868729 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e10de2-be2d-4409-be76-46f791703354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:30.870402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e10de2-be2d-4409-be76-46f791703354 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:30.870965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e10de2-be2d-4409-be76-46f791703354 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:30.871425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e10de2-be2d-4409-be76-46f791703354 admin admin] PATH_INFO: `/v3/projects/ddbb5bd19dbc45849abf82846237ec0a/users/91f576d404674e8491acfada7bbc15d0/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:30.874050 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 991/2011] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:30 2022] PUT /identity/v3/projects/aea4d2b8b59d46ed8f0c1c0ce42b6341/users/3c09acbd99fe4a1dacdd2c097c8f75d8/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:30.874671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482fc22-f17c-4a8d-bdeb-61493ef4b5e2 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.892554 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e10de2-be2d-4409-be76-46f791703354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddbb5bd19dbc45849abf82846237ec0a, user_id=91f576d404674e8491acfada7bbc15d0, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:30.895216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e10de2-be2d-4409-be76-46f791703354 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:30.919401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1021/2012] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:30 2022] PUT /identity/v3/projects/ddbb5bd19dbc45849abf82846237ec0a/users/91f576d404674e8491acfada7bbc15d0/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:30.920347 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e10de2-be2d-4409-be76-46f791703354 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:30.933571 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:30.956055 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482fc22-f17c-4a8d-bdeb-61493ef4b5e2 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:31.001112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e10de2-be2d-4409-be76-46f791703354 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:31.010024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c6b87-493c-4135-8572-02bb6e700662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:31.010934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c6b87-493c-4135-8572-02bb6e700662 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:31.011386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c6b87-493c-4135-8572-02bb6e700662 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:31.012094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c6b87-493c-4135-8572-02bb6e700662 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:31.013432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c6b87-493c-4135-8572-02bb6e700662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:31.019940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c6b87-493c-4135-8572-02bb6e700662 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:31.034093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1022/2013] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:31.034093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c6b87-493c-4135-8572-02bb6e700662 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:31.034908 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e95fb4-9e5f-4b8e-9b70-340f1ba65914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:31.035851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e95fb4-9e5f-4b8e-9b70-340f1ba65914 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:31.036132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e95fb4-9e5f-4b8e-9b70-340f1ba65914 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:31.036399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e95fb4-9e5f-4b8e-9b70-340f1ba65914 admin admin] PATH_INFO: `/v3/projects/7af8497b6bc7430da7420568576cfe6f/users/6c1423d81a6b45428008dffec9545d9d/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:31.043389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c6b87-493c-4135-8572-02bb6e700662 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:31.048417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1bf522-00a1-4737-98cc-ab036c85be12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:31.049091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bf522-00a1-4737-98cc-ab036c85be12 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:31.049375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bf522-00a1-4737-98cc-ab036c85be12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:31.049650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bf522-00a1-4737-98cc-ab036c85be12 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:31.050387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1bf522-00a1-4737-98cc-ab036c85be12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:31.052049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1bf522-00a1-4737-98cc-ab036c85be12 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:31.054079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e95fb4-9e5f-4b8e-9b70-340f1ba65914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af8497b6bc7430da7420568576cfe6f, user_id=6c1423d81a6b45428008dffec9545d9d, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:31.057595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e95fb4-9e5f-4b8e-9b70-340f1ba65914 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:31.065172 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1023/2014] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:31.069864 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1bf522-00a1-4737-98cc-ab036c85be12 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:31.076544 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 992/2015] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:30 2022] PUT /identity/v3/projects/7af8497b6bc7430da7420568576cfe6f/users/6c1423d81a6b45428008dffec9545d9d/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:31.077704 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e95fb4-9e5f-4b8e-9b70-340f1ba65914 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:31.084158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1bf522-00a1-4737-98cc-ab036c85be12 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:31.162982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e95fb4-9e5f-4b8e-9b70-340f1ba65914 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:31.168722 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba83c5-fc01-4cf6-b1e5-b20e8ae8e129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:31.169697 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba83c5-fc01-4cf6-b1e5-b20e8ae8e129 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:31.169910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba83c5-fc01-4cf6-b1e5-b20e8ae8e129 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:31.170184 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba83c5-fc01-4cf6-b1e5-b20e8ae8e129 admin admin] PATH_INFO: `/v3/projects/ddbb5bd19dbc45849abf82846237ec0a/users/91f576d404674e8491acfada7bbc15d0/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:31.172566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba83c5-fc01-4cf6-b1e5-b20e8ae8e129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddbb5bd19dbc45849abf82846237ec0a, user_id=91f576d404674e8491acfada7bbc15d0, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:31.172836 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed752e-706d-4639-b1b5-e97160b5943f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:31.173563 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed752e-706d-4639-b1b5-e97160b5943f admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:31.173850 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed752e-706d-4639-b1b5-e97160b5943f admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:31.174145 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed752e-706d-4639-b1b5-e97160b5943f admin admin] PATH_INFO: `/v3/projects/aea4d2b8b59d46ed8f0c1c0ce42b6341/users/3c09acbd99fe4a1dacdd2c097c8f75d8/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:31.175861 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba83c5-fc01-4cf6-b1e5-b20e8ae8e129 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:31.176198 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed752e-706d-4639-b1b5-e97160b5943f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea4d2b8b59d46ed8f0c1c0ce42b6341, user_id=3c09acbd99fe4a1dacdd2c097c8f75d8, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:31.179176 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed752e-706d-4639-b1b5-e97160b5943f admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:31.183698 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:31.188857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 993/2016] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:31 2022] PUT /identity/v3/projects/ddbb5bd19dbc45849abf82846237ec0a/users/91f576d404674e8491acfada7bbc15d0/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:31.190954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e745f-5b91-4414-b6e7-aeb622dce15c None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:31.191301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e745f-5b91-4414-b6e7-aeb622dce15c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:31.191678 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e745f-5b91-4414-b6e7-aeb622dce15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:31.194073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1024/2017] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:31 2022] PUT /identity/v3/projects/aea4d2b8b59d46ed8f0c1c0ce42b6341/users/3c09acbd99fe4a1dacdd2c097c8f75d8/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:31.200893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b00750-e62d-4572-82e7-94093003a3e5 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:31.201251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b00750-e62d-4572-82e7-94093003a3e5 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:31.201795 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b00750-e62d-4572-82e7-94093003a3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:31.228857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-753e745f-5b91-4414-b6e7-aeb622dce15c None None] MFA Rules not processed for user `6c1423d81a6b45428008dffec9545d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:31.230091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-753e745f-5b91-4414-b6e7-aeb622dce15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:31.234026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-f1b00750-e62d-4572-82e7-94093003a3e5 None None] MFA Rules not processed for user `91f576d404674e8491acfada7bbc15d0`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:31.235269 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f1b00750-e62d-4572-82e7-94093003a3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:31.308953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:31.314115 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1025/2018] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:31.315604 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb911887-efed-4522-ab80-e079bb80cb6a None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:31.315953 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb911887-efed-4522-ab80-e079bb80cb6a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:31.316215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb911887-efed-4522-ab80-e079bb80cb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:31.319419 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 994/2019] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:31.330243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e745f-5b91-4414-b6e7-aeb622dce15c None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:31.332633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-753e745f-5b91-4414-b6e7-aeb622dce15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:31.364275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-bb911887-efed-4522-ab80-e079bb80cb6a None None] MFA Rules not processed for user `3c09acbd99fe4a1dacdd2c097c8f75d8`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:31.364275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-bb911887-efed-4522-ab80-e079bb80cb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:31.440539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e745f-5b91-4414-b6e7-aeb622dce15c None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:31.444974 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7216380-2914-45b3-8630-d932984739fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:31.448203 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7216380-2914-45b3-8630-d932984739fd service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:31.448588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7216380-2914-45b3-8630-d932984739fd service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:31.448943 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7216380-2914-45b3-8630-d932984739fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:31.450838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1026/2020] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:31 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:31.451592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb911887-efed-4522-ab80-e079bb80cb6a None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:31.454283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-bb911887-efed-4522-ab80-e079bb80cb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:31.464714 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7216380-2914-45b3-8630-d932984739fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:31.466068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb911887-efed-4522-ab80-e079bb80cb6a None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:31.466358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7216380-2914-45b3-8630-d932984739fd service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:31.470459 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31100e1-9b3a-487b-ac71-b6a536d6e9fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:31.471119 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31100e1-9b3a-487b-ac71-b6a536d6e9fd service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:31.471406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31100e1-9b3a-487b-ac71-b6a536d6e9fd service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:31.473984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31100e1-9b3a-487b-ac71-b6a536d6e9fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:31.480299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 995/2021] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:53:31.484433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31100e1-9b3a-487b-ac71-b6a536d6e9fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:31.486196 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31100e1-9b3a-487b-ac71-b6a536d6e9fd service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:31.498522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1027/2022] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:31.525018 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7216380-2914-45b3-8630-d932984739fd service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:31.529649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31100e1-9b3a-487b-ac71-b6a536d6e9fd service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:31.537691 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7216380-2914-45b3-8630-d932984739fd service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:31.544351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bb17b-a73d-42d4-9133-8970dd903269 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:31.545955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bb17b-a73d-42d4-9133-8970dd903269 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:31.546953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bb17b-a73d-42d4-9133-8970dd903269 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:31.547369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bb17b-a73d-42d4-9133-8970dd903269 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:31.562597 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bb17b-a73d-42d4-9133-8970dd903269 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:31.565001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bb17b-a73d-42d4-9133-8970dd903269 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:31.582277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 996/2023] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:31 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:31.632050 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:31.649883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31100e1-9b3a-487b-ac71-b6a536d6e9fd service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:31.669279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677880c2-7589-424f-a735-76267f5f0e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:31.670281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677880c2-7589-424f-a735-76267f5f0e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:31.670640 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677880c2-7589-424f-a735-76267f5f0e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:31.671006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677880c2-7589-424f-a735-76267f5f0e0a admin admin] PATH_INFO: `/v3/users/6c1423d81a6b45428008dffec9545d9d` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:31.672580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677880c2-7589-424f-a735-76267f5f0e0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c1423d81a6b45428008dffec9545d9d)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:31.680903 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677880c2-7589-424f-a735-76267f5f0e0a admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:31.735247 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bb17b-a73d-42d4-9133-8970dd903269 service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:31.752705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bb17b-a73d-42d4-9133-8970dd903269 service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:31.769624 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c635d-723e-49ec-ba1c-e430987005af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:31.770498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c635d-723e-49ec-ba1c-e430987005af service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:31.770785 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c635d-723e-49ec-ba1c-e430987005af service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:31.771062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c635d-723e-49ec-ba1c-e430987005af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:31.805679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-677880c2-7589-424f-a735-76267f5f0e0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c1423d81a6b45428008dffec9545d9d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:31.808127 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-677880c2-7589-424f-a735-76267f5f0e0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c1423d81a6b45428008dffec9545d9d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:31.825298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-677880c2-7589-424f-a735-76267f5f0e0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c1423d81a6b45428008dffec9545d9d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:31.837955 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-677880c2-7589-424f-a735-76267f5f0e0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c1423d81a6b45428008dffec9545d9d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:31.851475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-677880c2-7589-424f-a735-76267f5f0e0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c1423d81a6b45428008dffec9545d9d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:31.866181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1028/2024] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:31 2022] DELETE /identity/v3/users/6c1423d81a6b45428008dffec9545d9d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:31.874920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac489c7c-bdcc-4b85-b8c6-42ec9764c0a3 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:31.876548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac489c7c-bdcc-4b85-b8c6-42ec9764c0a3 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:31.876797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c635d-723e-49ec-ba1c-e430987005af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:31.877343 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac489c7c-bdcc-4b85-b8c6-42ec9764c0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:31.878337 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c635d-723e-49ec-ba1c-e430987005af service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:31.889303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 997/2025] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:31.906820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-ac489c7c-bdcc-4b85-b8c6-42ec9764c0a3 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:31.908225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ac489c7c-bdcc-4b85-b8c6-42ec9764c0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:32.025935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1029/2026] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:32.036207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c635d-723e-49ec-ba1c-e430987005af service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:32.037686 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-931c635d-723e-49ec-ba1c-e430987005af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:32.157224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c635d-723e-49ec-ba1c-e430987005af service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:32.216366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:32.235313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8ac73-4add-4c21-8f98-e771de5e43cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:32.236673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8ac73-4add-4c21-8f98-e771de5e43cc service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:32.237009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8ac73-4add-4c21-8f98-e771de5e43cc service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:32.240041 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8ac73-4add-4c21-8f98-e771de5e43cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:32.257887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8ac73-4add-4c21-8f98-e771de5e43cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:32.260201 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8ac73-4add-4c21-8f98-e771de5e43cc service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:32.275834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 998/2027] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:32.413237 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac489c7c-bdcc-4b85-b8c6-42ec9764c0a3 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:32.419771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ac489c7c-bdcc-4b85-b8c6-42ec9764c0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:32.436490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac489c7c-bdcc-4b85-b8c6-42ec9764c0a3 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:32.449074 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54254c27-a79b-4879-bc63-f2a0a7494952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:32.450865 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54254c27-a79b-4879-bc63-f2a0a7494952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:32.450865 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54254c27-a79b-4879-bc63-f2a0a7494952 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:32.451296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54254c27-a79b-4879-bc63-f2a0a7494952 admin admin] PATH_INFO: `/v3/users/55bd0fb684514eabb4cd8345af2def6f` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:32.453266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54254c27-a79b-4879-bc63-f2a0a7494952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55bd0fb684514eabb4cd8345af2def6f)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:32.455553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54254c27-a79b-4879-bc63-f2a0a7494952 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:32.530534 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-54254c27-a79b-4879-bc63-f2a0a7494952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55bd0fb684514eabb4cd8345af2def6f'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:32.531426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-54254c27-a79b-4879-bc63-f2a0a7494952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55bd0fb684514eabb4cd8345af2def6f'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:32.556932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-54254c27-a79b-4879-bc63-f2a0a7494952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55bd0fb684514eabb4cd8345af2def6f'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:32.559858 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:32.567931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-54254c27-a79b-4879-bc63-f2a0a7494952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55bd0fb684514eabb4cd8345af2def6f'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:32.578748 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-54254c27-a79b-4879-bc63-f2a0a7494952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55bd0fb684514eabb4cd8345af2def6f'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:32.586327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1030/2028] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:32 2022] DELETE /identity/v3/users/55bd0fb684514eabb4cd8345af2def6f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:32.591700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a79753-e2a2-49e0-8f03-e506e0c017c4 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:32.592241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a79753-e2a2-49e0-8f03-e506e0c017c4 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:32.592443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a79753-e2a2-49e0-8f03-e506e0c017c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:32.613106 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-70a79753-e2a2-49e0-8f03-e506e0c017c4 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:32.614449 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-70a79753-e2a2-49e0-8f03-e506e0c017c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:32.639659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54254c27-a79b-4879-bc63-f2a0a7494952 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:32.641311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-54254c27-a79b-4879-bc63-f2a0a7494952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:32.723269 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 999/2029] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:32.739921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a79753-e2a2-49e0-8f03-e506e0c017c4 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:32.742774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54254c27-a79b-4879-bc63-f2a0a7494952 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:32.743261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-70a79753-e2a2-49e0-8f03-e506e0c017c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:32.748445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c192922-1484-432f-9413-b4f7a4def2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:32.749331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c192922-1484-432f-9413-b4f7a4def2f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:32.749622 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c192922-1484-432f-9413-b4f7a4def2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:32.749929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c192922-1484-432f-9413-b4f7a4def2f8 admin admin] PATH_INFO: `/v3/projects/7af8497b6bc7430da7420568576cfe6f` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:32.751220 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c192922-1484-432f-9413-b4f7a4def2f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7af8497b6bc7430da7420568576cfe6f)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:32.754211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c192922-1484-432f-9413-b4f7a4def2f8 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:32.825031 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7c192922-1484-432f-9413-b4f7a4def2f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7af8497b6bc7430da7420568576cfe6f', 'request_id': 'req-7c192922-1484-432f-9413-b4f7a4def2f8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:32.827551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a79753-e2a2-49e0-8f03-e506e0c017c4 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:32.836890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7c192922-1484-432f-9413-b4f7a4def2f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7af8497b6bc7430da7420568576cfe6f', 'request_id': 'req-7c192922-1484-432f-9413-b4f7a4def2f8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:32.849739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7c192922-1484-432f-9413-b4f7a4def2f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7af8497b6bc7430da7420568576cfe6f', 'request_id': 'req-7c192922-1484-432f-9413-b4f7a4def2f8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:32.879020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7c192922-1484-432f-9413-b4f7a4def2f8 admin admin] The token cache is being invalidate because project 7af8497b6bc7430da7420568576cfe6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:32.881366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7c192922-1484-432f-9413-b4f7a4def2f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:32.882736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7c192922-1484-432f-9413-b4f7a4def2f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:32.886967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1031/2030] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:32 2022] DELETE /identity/v3/projects/7af8497b6bc7430da7420568576cfe6f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:32.892405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c5b1c-728e-475e-9aaa-ab4226666741 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:32.892786 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c5b1c-728e-475e-9aaa-ab4226666741 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:32.893143 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c5b1c-728e-475e-9aaa-ab4226666741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:32.907366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9639e-a327-4c5c-834a-209ba2e77f48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:32.908240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9639e-a327-4c5c-834a-209ba2e77f48 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:32.908662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9639e-a327-4c5c-834a-209ba2e77f48 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:32.909388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9639e-a327-4c5c-834a-209ba2e77f48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:32.913085 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7dd9639e-a327-4c5c-834a-209ba2e77f48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:32.941982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-1b8c5b1c-728e-475e-9aaa-ab4226666741 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:32.943583 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1b8c5b1c-728e-475e-9aaa-ab4226666741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:33.008969 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:33.036878 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9639e-a327-4c5c-834a-209ba2e77f48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:33.038336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9639e-a327-4c5c-834a-209ba2e77f48 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:33.052629 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1000/2031] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:53:33.063881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1032/2032] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:33.072248 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99d709-ac25-43dd-a555-bc131bd2e0f6 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:33.072806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99d709-ac25-43dd-a555-bc131bd2e0f6 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:33.073167 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99d709-ac25-43dd-a555-bc131bd2e0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:33.093044 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-0a99d709-ac25-43dd-a555-bc131bd2e0f6 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:33.096807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-0a99d709-ac25-43dd-a555-bc131bd2e0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:33.107372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1001/2033] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:33.117858 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c5b1c-728e-475e-9aaa-ab4226666741 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:33.131390 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c5b1c-728e-475e-9aaa-ab4226666741 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:33.137762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38196ed-5253-45b3-97ad-f08e451209b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:33.138405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38196ed-5253-45b3-97ad-f08e451209b4 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:33.138696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38196ed-5253-45b3-97ad-f08e451209b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:33.138972 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38196ed-5253-45b3-97ad-f08e451209b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:33.139796 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38196ed-5253-45b3-97ad-f08e451209b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:53:33.140118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38196ed-5253-45b3-97ad-f08e451209b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:33.142172 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38196ed-5253-45b3-97ad-f08e451209b4 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:33.171475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1033/2034] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:53:33.176351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99d709-ac25-43dd-a555-bc131bd2e0f6 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:33.198427 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99d709-ac25-43dd-a555-bc131bd2e0f6 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:33.210960 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a3760-6297-43cd-896a-bef603a5040f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:33.212509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a3760-6297-43cd-896a-bef603a5040f admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:33.212805 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a3760-6297-43cd-896a-bef603a5040f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:33.213083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a3760-6297-43cd-896a-bef603a5040f admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:33.213839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a3760-6297-43cd-896a-bef603a5040f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:33.215822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a3760-6297-43cd-896a-bef603a5040f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:33.250761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1002/2035] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:33 2022] POST /identity/v3/projects => generated 375 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:53:33.256765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38196ed-5253-45b3-97ad-f08e451209b4 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:33.267879 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:33.271271 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38196ed-5253-45b3-97ad-f08e451209b4 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:33.380376 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf228a1-dea5-489b-ab3d-de975935b543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:33.382058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf228a1-dea5-489b-ab3d-de975935b543 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:33.382058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf228a1-dea5-489b-ab3d-de975935b543 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:33.382058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf228a1-dea5-489b-ab3d-de975935b543 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:33.382323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf228a1-dea5-489b-ab3d-de975935b543 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:33.385472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf228a1-dea5-489b-ab3d-de975935b543 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:33.427755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1034/2036] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:33 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:33.431776 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a3760-6297-43cd-896a-bef603a5040f admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:33.448617 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a3760-6297-43cd-896a-bef603a5040f admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:33.455322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3ef02-04ed-4354-b3f4-e0d33f06f967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:33.456566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3ef02-04ed-4354-b3f4-e0d33f06f967 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:33.457300 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3ef02-04ed-4354-b3f4-e0d33f06f967 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:33.458017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3ef02-04ed-4354-b3f4-e0d33f06f967 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:33.459297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3ef02-04ed-4354-b3f4-e0d33f06f967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:33.462373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3ef02-04ed-4354-b3f4-e0d33f06f967 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:33.479758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1003/2037] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:53:33.484102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf228a1-dea5-489b-ab3d-de975935b543 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:33.502264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf228a1-dea5-489b-ab3d-de975935b543 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:33.515022 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3ef02-04ed-4354-b3f4-e0d33f06f967 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:33.521399 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da07cf-05d8-4625-9324-e467bcf9996b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:33.521542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-58d3ef02-04ed-4354-b3f4-e0d33f06f967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:33.524820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da07cf-05d8-4625-9324-e467bcf9996b admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:33.525810 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da07cf-05d8-4625-9324-e467bcf9996b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:33.526275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da07cf-05d8-4625-9324-e467bcf9996b admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:33.528984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da07cf-05d8-4625-9324-e467bcf9996b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:33.531444 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da07cf-05d8-4625-9324-e467bcf9996b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:33.541619 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3ef02-04ed-4354-b3f4-e0d33f06f967 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:33.556078 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:33.556945 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:33.557226 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:33.563603 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002 admin admin] PATH_INFO: `/v3/projects/e4c00779f3c5422ca29a17f07e97498b` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:33.565147 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4c00779f3c5422ca29a17f07e97498b)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:33.569138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1035/2038] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:33.572855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:33.576867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da07cf-05d8-4625-9324-e467bcf9996b admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:33.582595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b1da07cf-05d8-4625-9324-e467bcf9996b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:33.616808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da07cf-05d8-4625-9324-e467bcf9996b admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:33.637799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4672fd-c033-40ae-abfb-8755a13bea8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:33.638743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4672fd-c033-40ae-abfb-8755a13bea8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:33.639037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4672fd-c033-40ae-abfb-8755a13bea8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:33.641431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4672fd-c033-40ae-abfb-8755a13bea8a admin admin] PATH_INFO: `/v3/users/df4d94dccf564514b2d450d11775466e` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:33.644578 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4672fd-c033-40ae-abfb-8755a13bea8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=df4d94dccf564514b2d450d11775466e)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:33.650514 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4672fd-c033-40ae-abfb-8755a13bea8a admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:33.718565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4c00779f3c5422ca29a17f07e97498b', 'request_id': 'req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:33.742103 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4c00779f3c5422ca29a17f07e97498b', 'request_id': 'req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:33.753013 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4c00779f3c5422ca29a17f07e97498b', 'request_id': 'req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:33.761200 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-fd4672fd-c033-40ae-abfb-8755a13bea8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df4d94dccf564514b2d450d11775466e'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:33.765704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-fd4672fd-c033-40ae-abfb-8755a13bea8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df4d94dccf564514b2d450d11775466e'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:33.770592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002 admin admin] The token cache is being invalidate because project e4c00779f3c5422ca29a17f07e97498b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:33.770998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:33.772503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:33.775828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1004/2039] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:33 2022] DELETE /identity/v3/projects/e4c00779f3c5422ca29a17f07e97498b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:33.776888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:33.781302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:33.796388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-fd4672fd-c033-40ae-abfb-8755a13bea8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df4d94dccf564514b2d450d11775466e'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:33.819253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-fd4672fd-c033-40ae-abfb-8755a13bea8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df4d94dccf564514b2d450d11775466e'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:33.834692 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-fd4672fd-c033-40ae-abfb-8755a13bea8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df4d94dccf564514b2d450d11775466e'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:33.848764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1036/2040] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:33 2022] DELETE /identity/v3/users/df4d94dccf564514b2d450d11775466e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:33.850218 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c8704-c726-4d04-bb5d-ec9612f02cf9 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:33.850510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c8704-c726-4d04-bb5d-ec9612f02cf9 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:33.850793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c8704-c726-4d04-bb5d-ec9612f02cf9 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:33.853266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1037/2041] 10.222.0.60 () {54 vars in 921 bytes} [Thu Jul 21 20:53:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:53:33.854201 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c8704-c726-4d04-bb5d-ec9612f02cf9 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:33.856509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ef9c8704-c726-4d04-bb5d-ec9612f02cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:33.880737 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d6a7d-f4ef-4d40-af49-7d9783b66002 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:33.939343 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:33.958794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c8704-c726-4d04-bb5d-ec9612f02cf9 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:33.963694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:33.969882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc08cb0-2d80-469b-8758-f40a2e49e568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:33.970819 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc08cb0-2d80-469b-8758-f40a2e49e568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:33.971111 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc08cb0-2d80-469b-8758-f40a2e49e568 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:33.971403 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc08cb0-2d80-469b-8758-f40a2e49e568 admin admin] PATH_INFO: `/v3/users/91f576d404674e8491acfada7bbc15d0` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:33.980850 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc08cb0-2d80-469b-8758-f40a2e49e568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91f576d404674e8491acfada7bbc15d0)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:33.980850 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc08cb0-2d80-469b-8758-f40a2e49e568 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:33.997716 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7f9cf-be3f-48f7-be3d-087f0f5a775d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:33.998739 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7f9cf-be3f-48f7-be3d-087f0f5a775d admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:33.999020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7f9cf-be3f-48f7-be3d-087f0f5a775d admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:33.999294 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7f9cf-be3f-48f7-be3d-087f0f5a775d admin admin] PATH_INFO: `/v3/projects/574b27bf902648ebade2384ff1071d57/users/ad7325a851ee482b8ab61b70ffa92bfe/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:34.020059 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7f9cf-be3f-48f7-be3d-087f0f5a775d admin admin] RBAC: Authorizing `identity:create_grant(project_id=574b27bf902648ebade2384ff1071d57, user_id=ad7325a851ee482b8ab61b70ffa92bfe, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:34.022998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7f9cf-be3f-48f7-be3d-087f0f5a775d admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:34.043184 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-6cc08cb0-2d80-469b-8758-f40a2e49e568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91f576d404674e8491acfada7bbc15d0'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:34.043314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1005/2042] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:33 2022] PUT /identity/v3/projects/574b27bf902648ebade2384ff1071d57/users/ad7325a851ee482b8ab61b70ffa92bfe/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:34.044033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-6cc08cb0-2d80-469b-8758-f40a2e49e568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91f576d404674e8491acfada7bbc15d0'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:34.044653 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4893149-8d73-4322-aa28-c0aa25b90349 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:34.044941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4893149-8d73-4322-aa28-c0aa25b90349 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:34.045262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4893149-8d73-4322-aa28-c0aa25b90349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:34.072094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-6cc08cb0-2d80-469b-8758-f40a2e49e568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91f576d404674e8491acfada7bbc15d0'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:34.080512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c4893149-8d73-4322-aa28-c0aa25b90349 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:34.081766 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c4893149-8d73-4322-aa28-c0aa25b90349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:34.083068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-6cc08cb0-2d80-469b-8758-f40a2e49e568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91f576d404674e8491acfada7bbc15d0'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:34.093873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-6cc08cb0-2d80-469b-8758-f40a2e49e568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91f576d404674e8491acfada7bbc15d0'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:34.101228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1038/2043] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:33 2022] DELETE /identity/v3/users/91f576d404674e8491acfada7bbc15d0 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:34.102890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1137399-0a17-44ce-b524-38be8b46a929 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:34.103535 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1137399-0a17-44ce-b524-38be8b46a929 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:34.103535 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1137399-0a17-44ce-b524-38be8b46a929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:34.140177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-a1137399-0a17-44ce-b524-38be8b46a929 None None] MFA Rules not processed for user `77812b356c974d2b80102217956169db`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:34.143375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a1137399-0a17-44ce-b524-38be8b46a929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:34.173206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1006/2044] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:34.174721 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f04cbb-09bf-44ed-9bb9-d2e835fb40cf None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:34.175156 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f04cbb-09bf-44ed-9bb9-d2e835fb40cf None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:34.175622 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f04cbb-09bf-44ed-9bb9-d2e835fb40cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:34.240580 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-75f04cbb-09bf-44ed-9bb9-d2e835fb40cf None None] MFA Rules not processed for user `ad7325a851ee482b8ab61b70ffa92bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:34.241984 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-75f04cbb-09bf-44ed-9bb9-d2e835fb40cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:34.259685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1039/2045] 10.222.0.60 () {58 vars in 1018 bytes} [Thu Jul 21 20:53:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:34.260211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1137399-0a17-44ce-b524-38be8b46a929 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:34.277330 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a1137399-0a17-44ce-b524-38be8b46a929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:34.372862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1007/2046] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:34.373870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f04cbb-09bf-44ed-9bb9-d2e835fb40cf None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:34.375381 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-75f04cbb-09bf-44ed-9bb9-d2e835fb40cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:34.387467 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1137399-0a17-44ce-b524-38be8b46a929 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:34.392872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788cfd8-ef5d-4280-b9ae-02e93fb587e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:34.393742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788cfd8-ef5d-4280-b9ae-02e93fb587e3 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:34.394037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788cfd8-ef5d-4280-b9ae-02e93fb587e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:34.394322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788cfd8-ef5d-4280-b9ae-02e93fb587e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:34.447079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:34.493881 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:34.507168 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f04cbb-09bf-44ed-9bb9-d2e835fb40cf None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:34.513963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0ad0c-8366-4994-95aa-3ef35d86c9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:34.515075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0ad0c-8366-4994-95aa-3ef35d86c9e0 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:34.517307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0ad0c-8366-4994-95aa-3ef35d86c9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:34.517799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0ad0c-8366-4994-95aa-3ef35d86c9e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:34.518652 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0ad0c-8366-4994-95aa-3ef35d86c9e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:34.521064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0ad0c-8366-4994-95aa-3ef35d86c9e0 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:34.527513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788cfd8-ef5d-4280-b9ae-02e93fb587e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:34.529262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788cfd8-ef5d-4280-b9ae-02e93fb587e3 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:34.544844 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1040/2047] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:34.546164 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788cfd8-ef5d-4280-b9ae-02e93fb587e3 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:34.547673 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9788cfd8-ef5d-4280-b9ae-02e93fb587e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:34.565897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1008/2048] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:34 2022] POST /identity/v3/projects => generated 375 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:34.570305 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0ad0c-8366-4994-95aa-3ef35d86c9e0 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:34.571539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788cfd8-ef5d-4280-b9ae-02e93fb587e3 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:34.711151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0ad0c-8366-4994-95aa-3ef35d86c9e0 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:34.719847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7b2df-5a05-45c5-862a-581b50bef520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:34.721006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7b2df-5a05-45c5-862a-581b50bef520 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:34.721307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566462c2-b14a-4be9-89ef-79b0c40e3e0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:34.721719 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7b2df-5a05-45c5-862a-581b50bef520 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:34.721988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566462c2-b14a-4be9-89ef-79b0c40e3e0d service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:34.722107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7b2df-5a05-45c5-862a-581b50bef520 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:34.722325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566462c2-b14a-4be9-89ef-79b0c40e3e0d service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:34.722561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566462c2-b14a-4be9-89ef-79b0c40e3e0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:34.723993 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7b2df-5a05-45c5-862a-581b50bef520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:34.729257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7b2df-5a05-45c5-862a-581b50bef520 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:34.750463 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566462c2-b14a-4be9-89ef-79b0c40e3e0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:34.752392 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566462c2-b14a-4be9-89ef-79b0c40e3e0d service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:34.770740 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1041/2049] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:34.773694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1009/2050] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:34 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:34.778406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566462c2-b14a-4be9-89ef-79b0c40e3e0d service nova] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:34.814557 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566462c2-b14a-4be9-89ef-79b0c40e3e0d service nova] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:34.834472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68588d-94cf-4e17-9362-0da0cc8651b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:34.835780 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68588d-94cf-4e17-9362-0da0cc8651b3 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:34.836293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68588d-94cf-4e17-9362-0da0cc8651b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:34.838386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68588d-94cf-4e17-9362-0da0cc8651b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:34.839344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68588d-94cf-4e17-9362-0da0cc8651b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:34.842252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68588d-94cf-4e17-9362-0da0cc8651b3 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:34.861763 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1042/2051] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:34.863578 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7b2df-5a05-45c5-862a-581b50bef520 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:34.884420 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7b2df-5a05-45c5-862a-581b50bef520 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:34.892983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1d854-5b98-4cc1-a7ea-b42bb289f605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:34.894008 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1d854-5b98-4cc1-a7ea-b42bb289f605 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:34.895090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1d854-5b98-4cc1-a7ea-b42bb289f605 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:34.895470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1d854-5b98-4cc1-a7ea-b42bb289f605 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:34.896809 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1d854-5b98-4cc1-a7ea-b42bb289f605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:34.899965 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1d854-5b98-4cc1-a7ea-b42bb289f605 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:34.935701 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1010/2052] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:34.941489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68588d-94cf-4e17-9362-0da0cc8651b3 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:34.957627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68588d-94cf-4e17-9362-0da0cc8651b3 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:34.965732 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12671e48-19d3-4f91-891c-098a20d0c742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:34.966666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12671e48-19d3-4f91-891c-098a20d0c742 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:34.966952 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12671e48-19d3-4f91-891c-098a20d0c742 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:34.967231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12671e48-19d3-4f91-891c-098a20d0c742 admin admin] PATH_INFO: `/v3/projects/cdc87125a7f8474f8907abb35ee0c0e8/users/a723f0f227624405ad449c7d8bf6bd3e/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:34.988187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12671e48-19d3-4f91-891c-098a20d0c742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc87125a7f8474f8907abb35ee0c0e8, user_id=a723f0f227624405ad449c7d8bf6bd3e, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:34.991149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12671e48-19d3-4f91-891c-098a20d0c742 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:35.015756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1043/2053] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:34 2022] PUT /identity/v3/projects/cdc87125a7f8474f8907abb35ee0c0e8/users/a723f0f227624405ad449c7d8bf6bd3e/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:35.025981 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167d005-85bc-48bd-bbc8-6942acb81fbc None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:35.026372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167d005-85bc-48bd-bbc8-6942acb81fbc None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:35.026881 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167d005-85bc-48bd-bbc8-6942acb81fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:35.091552 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-5167d005-85bc-48bd-bbc8-6942acb81fbc None None] MFA Rules not processed for user `a723f0f227624405ad449c7d8bf6bd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:35.095828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5167d005-85bc-48bd-bbc8-6942acb81fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:35.162075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12671e48-19d3-4f91-891c-098a20d0c742 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:35.163970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-12671e48-19d3-4f91-891c-098a20d0c742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:35.256489 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1011/2054] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:35.263074 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167d005-85bc-48bd-bbc8-6942acb81fbc None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:35.265237 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5167d005-85bc-48bd-bbc8-6942acb81fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:35.319454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12671e48-19d3-4f91-891c-098a20d0c742 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:35.326285 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1baa7-3176-4a22-80a9-457d6db8b93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:35.327602 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1baa7-3176-4a22-80a9-457d6db8b93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:35.328079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1baa7-3176-4a22-80a9-457d6db8b93b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:35.330000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1baa7-3176-4a22-80a9-457d6db8b93b admin admin] PATH_INFO: `/v3/projects/8877aa18458245259526a2d457a6040d` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:35.331618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1baa7-3176-4a22-80a9-457d6db8b93b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8877aa18458245259526a2d457a6040d)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:35.333705 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1baa7-3176-4a22-80a9-457d6db8b93b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:35.343029 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:35.365733 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:35.385288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167d005-85bc-48bd-bbc8-6942acb81fbc None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:35.393611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f4163-c0aa-449a-b97f-421a0ea220e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:35.394646 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f4163-c0aa-449a-b97f-421a0ea220e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:35.395083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f4163-c0aa-449a-b97f-421a0ea220e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:35.395431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f4163-c0aa-449a-b97f-421a0ea220e4 admin admin] PATH_INFO: `/v3/projects/ddbb5bd19dbc45849abf82846237ec0a` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:35.396901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f4163-c0aa-449a-b97f-421a0ea220e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddbb5bd19dbc45849abf82846237ec0a)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:35.401712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f4163-c0aa-449a-b97f-421a0ea220e4 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:35.443156 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-85a1baa7-3176-4a22-80a9-457d6db8b93b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8877aa18458245259526a2d457a6040d', 'request_id': 'req-85a1baa7-3176-4a22-80a9-457d6db8b93b'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:35.451244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-85a1baa7-3176-4a22-80a9-457d6db8b93b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8877aa18458245259526a2d457a6040d', 'request_id': 'req-85a1baa7-3176-4a22-80a9-457d6db8b93b'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:35.458608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-85a1baa7-3176-4a22-80a9-457d6db8b93b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8877aa18458245259526a2d457a6040d', 'request_id': 'req-85a1baa7-3176-4a22-80a9-457d6db8b93b'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:35.477076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-475f4163-c0aa-449a-b97f-421a0ea220e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddbb5bd19dbc45849abf82846237ec0a', 'request_id': 'req-475f4163-c0aa-449a-b97f-421a0ea220e4'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:35.477806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-85a1baa7-3176-4a22-80a9-457d6db8b93b admin admin] The token cache is being invalidate because project 8877aa18458245259526a2d457a6040d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:35.478270 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-85a1baa7-3176-4a22-80a9-457d6db8b93b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:35.479064 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-85a1baa7-3176-4a22-80a9-457d6db8b93b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:35.481159 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1044/2055] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:35 2022] DELETE /identity/v3/projects/8877aa18458245259526a2d457a6040d => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:35.482464 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1baa7-3176-4a22-80a9-457d6db8b93b admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:35.485562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-85a1baa7-3176-4a22-80a9-457d6db8b93b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:35.490159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-475f4163-c0aa-449a-b97f-421a0ea220e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddbb5bd19dbc45849abf82846237ec0a', 'request_id': 'req-475f4163-c0aa-449a-b97f-421a0ea220e4'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:35.499113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-475f4163-c0aa-449a-b97f-421a0ea220e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddbb5bd19dbc45849abf82846237ec0a', 'request_id': 'req-475f4163-c0aa-449a-b97f-421a0ea220e4'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:35.519668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-475f4163-c0aa-449a-b97f-421a0ea220e4 admin admin] The token cache is being invalidate because project ddbb5bd19dbc45849abf82846237ec0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:53:35.520034 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-475f4163-c0aa-449a-b97f-421a0ea220e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:35.521253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-475f4163-c0aa-449a-b97f-421a0ea220e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:35.527288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1012/2056] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:35 2022] DELETE /identity/v3/projects/ddbb5bd19dbc45849abf82846237ec0a => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:35.527288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191ee9b-c789-49c4-8972-84068c2c77bd None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:35.527288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191ee9b-c789-49c4-8972-84068c2c77bd None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:35.527288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191ee9b-c789-49c4-8972-84068c2c77bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:35.553263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-e191ee9b-c789-49c4-8972-84068c2c77bd None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:35.554860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e191ee9b-c789-49c4-8972-84068c2c77bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:35.588618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1baa7-3176-4a22-80a9-457d6db8b93b admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:35.664963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1013/2057] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:35.666813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae66d7-4bb1-4dfd-ad5b-ddb74be1f2b6 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:35.667172 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae66d7-4bb1-4dfd-ad5b-ddb74be1f2b6 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:35.667523 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae66d7-4bb1-4dfd-ad5b-ddb74be1f2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:35.688311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-30ae66d7-4bb1-4dfd-ad5b-ddb74be1f2b6 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:35.689653 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-30ae66d7-4bb1-4dfd-ad5b-ddb74be1f2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:35.699269 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1014/2058] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:35.708569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd8c21-b9e2-427b-ba65-1da2547f13cf None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:35.708953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd8c21-b9e2-427b-ba65-1da2547f13cf None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:35.709195 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7b775-84ad-40c2-9d45-abec04f4766f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:35.709354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd8c21-b9e2-427b-ba65-1da2547f13cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:35.710240 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7b775-84ad-40c2-9d45-abec04f4766f admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:35.710875 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7b775-84ad-40c2-9d45-abec04f4766f admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:35.712373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7b775-84ad-40c2-9d45-abec04f4766f admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:35.713521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7b775-84ad-40c2-9d45-abec04f4766f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:35.716396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7b775-84ad-40c2-9d45-abec04f4766f admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:35.730612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-5bdd8c21-b9e2-427b-ba65-1da2547f13cf None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:35.732080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5bdd8c21-b9e2-427b-ba65-1da2547f13cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:35.749740 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1015/2059] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:35.751565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e263b17-fd19-4f4c-9997-f98850cf3879 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:35.752001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e263b17-fd19-4f4c-9997-f98850cf3879 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:35.752422 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e263b17-fd19-4f4c-9997-f98850cf3879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:35.752793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1045/2060] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:35 2022] POST /identity/v3/projects => generated 375 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:35.756523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7b775-84ad-40c2-9d45-abec04f4766f admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:35.782172 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-7e263b17-fd19-4f4c-9997-f98850cf3879 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:35.784302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7e263b17-fd19-4f4c-9997-f98850cf3879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:35.801222 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:35.870255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7b775-84ad-40c2-9d45-abec04f4766f admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:35.876730 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83419c8-044c-46e0-9a82-f0b5b09388a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:35.877956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83419c8-044c-46e0-9a82-f0b5b09388a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:35.878623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83419c8-044c-46e0-9a82-f0b5b09388a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:35.879618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83419c8-044c-46e0-9a82-f0b5b09388a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:35.880619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83419c8-044c-46e0-9a82-f0b5b09388a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:53:35.880995 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83419c8-044c-46e0-9a82-f0b5b09388a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:35.882714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83419c8-044c-46e0-9a82-f0b5b09388a8 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:35.888335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1016/2061] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:35.889272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e263b17-fd19-4f4c-9997-f98850cf3879 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:35.890655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7e263b17-fd19-4f4c-9997-f98850cf3879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:35.907119 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1046/2062] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:53:35.909262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83419c8-044c-46e0-9a82-f0b5b09388a8 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:35.912002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e263b17-fd19-4f4c-9997-f98850cf3879 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:35.923094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fc8e2-cd2b-417b-8133-fb7f2bf002ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:35.924583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fc8e2-cd2b-417b-8133-fb7f2bf002ba admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:35.925090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fc8e2-cd2b-417b-8133-fb7f2bf002ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:35.925690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fc8e2-cd2b-417b-8133-fb7f2bf002ba admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:35.926759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738fc8e2-cd2b-417b-8133-fb7f2bf002ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:35.927744 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83419c8-044c-46e0-9a82-f0b5b09388a8 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:35.929625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738fc8e2-cd2b-417b-8133-fb7f2bf002ba admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:35.934235 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591fd81-7ddc-4aff-928f-ad38f99c59d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:35.935372 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591fd81-7ddc-4aff-928f-ad38f99c59d6 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:35.935926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591fd81-7ddc-4aff-928f-ad38f99c59d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:35.936428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591fd81-7ddc-4aff-928f-ad38f99c59d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:35.938175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591fd81-7ddc-4aff-928f-ad38f99c59d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:53:35.938546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591fd81-7ddc-4aff-928f-ad38f99c59d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:35.940876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591fd81-7ddc-4aff-928f-ad38f99c59d6 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:35.942031 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:35.960547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1047/2063] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:53:35.960547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591fd81-7ddc-4aff-928f-ad38f99c59d6 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:35.968129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1017/2064] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:35 2022] POST /identity/v3/users => generated 294 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:35.969444 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fc8e2-cd2b-417b-8133-fb7f2bf002ba admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:35.971313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591fd81-7ddc-4aff-928f-ad38f99c59d6 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:35.977869 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902ff63-cbaf-44cb-9605-ce8408168165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:35.979752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902ff63-cbaf-44cb-9605-ce8408168165 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:35.979752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902ff63-cbaf-44cb-9605-ce8408168165 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:35.979752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902ff63-cbaf-44cb-9605-ce8408168165 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:35.980029 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902ff63-cbaf-44cb-9605-ce8408168165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:35.981864 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902ff63-cbaf-44cb-9605-ce8408168165 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:35.986755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fc8e2-cd2b-417b-8133-fb7f2bf002ba admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:35.993742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebe256-8586-41a9-ad5d-54274a312182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:35.994410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebe256-8586-41a9-ad5d-54274a312182 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:35.995053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebe256-8586-41a9-ad5d-54274a312182 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:35.995053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebe256-8586-41a9-ad5d-54274a312182 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:35.995651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ebe256-8586-41a9-ad5d-54274a312182 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:35.997589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ebe256-8586-41a9-ad5d-54274a312182 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:36.026301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1048/2065] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:35 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:36.027107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1018/2066] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:53:35 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:36.027949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebe256-8586-41a9-ad5d-54274a312182 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:36.028075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902ff63-cbaf-44cb-9605-ce8408168165 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:36.108703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902ff63-cbaf-44cb-9605-ce8408168165 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:36.112708 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebe256-8586-41a9-ad5d-54274a312182 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:36.114140 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab7cce-bf80-45b4-9ed2-d3bcf61693ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:36.114822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab7cce-bf80-45b4-9ed2-d3bcf61693ce admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.115106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab7cce-bf80-45b4-9ed2-d3bcf61693ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.115383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab7cce-bf80-45b4-9ed2-d3bcf61693ce admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.116232 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab7cce-bf80-45b4-9ed2-d3bcf61693ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:36.118345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab7cce-bf80-45b4-9ed2-d3bcf61693ce admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:36.118558 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574cf144-e04c-48a3-bff2-d239ad53138d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:36.119238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cf144-e04c-48a3-bff2-d239ad53138d admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.119526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cf144-e04c-48a3-bff2-d239ad53138d admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.119950 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cf144-e04c-48a3-bff2-d239ad53138d admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.121035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574cf144-e04c-48a3-bff2-d239ad53138d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:36.122563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574cf144-e04c-48a3-bff2-d239ad53138d admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:36.129476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1049/2067] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:36.130499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab7cce-bf80-45b4-9ed2-d3bcf61693ce admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:36.145241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1019/2068] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:36 2022] POST /identity/v3/users => generated 284 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:36.146512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574cf144-e04c-48a3-bff2-d239ad53138d admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:36.147365 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab7cce-bf80-45b4-9ed2-d3bcf61693ce admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:36.153136 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26984d-6a51-4f4c-a747-0389e0067d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:36.153929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26984d-6a51-4f4c-a747-0389e0067d9c admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.154384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26984d-6a51-4f4c-a747-0389e0067d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.154803 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26984d-6a51-4f4c-a747-0389e0067d9c admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.155854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26984d-6a51-4f4c-a747-0389e0067d9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:36.159145 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26984d-6a51-4f4c-a747-0389e0067d9c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:36.162657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574cf144-e04c-48a3-bff2-d239ad53138d admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:36.168937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc764654-7571-4937-8227-06063d696b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:36.169873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc764654-7571-4937-8227-06063d696b0a admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.170154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc764654-7571-4937-8227-06063d696b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.170432 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc764654-7571-4937-8227-06063d696b0a admin admin] PATH_INFO: `/v3/projects/fae906fc3738448cbf57c25c264e25ac/users/34a21df0ef8e40258a5626ec71f1ae9e/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.185813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1050/2069] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:53:36 2022] POST /identity/v3/users => generated 293 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:53:36.188638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26984d-6a51-4f4c-a747-0389e0067d9c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:36.189779 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc764654-7571-4937-8227-06063d696b0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fae906fc3738448cbf57c25c264e25ac, user_id=34a21df0ef8e40258a5626ec71f1ae9e, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:36.192496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc764654-7571-4937-8227-06063d696b0a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:36.201391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26984d-6a51-4f4c-a747-0389e0067d9c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:36.206510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c75d86-ecb2-4a1a-859e-d6a2162daa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:36.207231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c75d86-ecb2-4a1a-859e-d6a2162daa8b admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.207497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c75d86-ecb2-4a1a-859e-d6a2162daa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.207702 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c75d86-ecb2-4a1a-859e-d6a2162daa8b admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.208477 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c75d86-ecb2-4a1a-859e-d6a2162daa8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:36.210075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1020/2070] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:36 2022] PUT /identity/v3/projects/fae906fc3738448cbf57c25c264e25ac/users/34a21df0ef8e40258a5626ec71f1ae9e/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:36.210258 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c75d86-ecb2-4a1a-859e-d6a2162daa8b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:36.210908 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc764654-7571-4937-8227-06063d696b0a admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:36.224856 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1051/2071] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:36.225704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c75d86-ecb2-4a1a-859e-d6a2162daa8b admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:36.236620 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c75d86-ecb2-4a1a-859e-d6a2162daa8b admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:36.285157 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc764654-7571-4937-8227-06063d696b0a admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:36.290760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd257a1-5693-4cd9-886b-c3437db29e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:36.291475 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd257a1-5693-4cd9-886b-c3437db29e48 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.291702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd257a1-5693-4cd9-886b-c3437db29e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.292020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd257a1-5693-4cd9-886b-c3437db29e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.292148 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:36.292765 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd257a1-5693-4cd9-886b-c3437db29e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:36.294496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd257a1-5693-4cd9-886b-c3437db29e48 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:36.305859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0065d-dda1-41d7-a403-0709cc335a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:36.306695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1021/2072] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:36.306804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0065d-dda1-41d7-a403-0709cc335a98 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.307026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0065d-dda1-41d7-a403-0709cc335a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.307301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0065d-dda1-41d7-a403-0709cc335a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.308077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0065d-dda1-41d7-a403-0709cc335a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:36.308849 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd257a1-5693-4cd9-886b-c3437db29e48 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:36.309817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0065d-dda1-41d7-a403-0709cc335a98 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:36.319362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd257a1-5693-4cd9-886b-c3437db29e48 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:36.320537 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1052/2073] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:36.321631 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0065d-dda1-41d7-a403-0709cc335a98 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:36.328967 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866c1ac-f222-4755-b4b3-10b5e0d27939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:36.329649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866c1ac-f222-4755-b4b3-10b5e0d27939 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.329936 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866c1ac-f222-4755-b4b3-10b5e0d27939 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.330343 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866c1ac-f222-4755-b4b3-10b5e0d27939 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.331083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866c1ac-f222-4755-b4b3-10b5e0d27939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:36.331526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0065d-dda1-41d7-a403-0709cc335a98 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:36.332727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866c1ac-f222-4755-b4b3-10b5e0d27939 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:36.336257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870f49f-fe5a-4f1c-a586-44c65beec52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:36.337289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870f49f-fe5a-4f1c-a586-44c65beec52e admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.337578 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870f49f-fe5a-4f1c-a586-44c65beec52e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.337859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870f49f-fe5a-4f1c-a586-44c65beec52e admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.338595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870f49f-fe5a-4f1c-a586-44c65beec52e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:36.340213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2870f49f-fe5a-4f1c-a586-44c65beec52e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:36.343583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1022/2074] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:36.344576 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866c1ac-f222-4755-b4b3-10b5e0d27939 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:36.350931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1053/2075] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:53:36.352433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870f49f-fe5a-4f1c-a586-44c65beec52e admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:36.358821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866c1ac-f222-4755-b4b3-10b5e0d27939 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:36.363666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870f49f-fe5a-4f1c-a586-44c65beec52e admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:36.364322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f990dcf-0e9c-4654-a7df-a36924432662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:36.365221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f990dcf-0e9c-4654-a7df-a36924432662 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.365604 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f990dcf-0e9c-4654-a7df-a36924432662 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.365839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f990dcf-0e9c-4654-a7df-a36924432662 admin admin] PATH_INFO: `/v3/projects/fae906fc3738448cbf57c25c264e25ac/users/34a21df0ef8e40258a5626ec71f1ae9e/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.368961 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f990dcf-0e9c-4654-a7df-a36924432662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fae906fc3738448cbf57c25c264e25ac, user_id=34a21df0ef8e40258a5626ec71f1ae9e, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:36.369326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498792b-a753-41ee-9a7f-2d502c52343d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:36.370216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498792b-a753-41ee-9a7f-2d502c52343d admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.370499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498792b-a753-41ee-9a7f-2d502c52343d admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.370774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498792b-a753-41ee-9a7f-2d502c52343d admin admin] PATH_INFO: `/v3/projects/71b5f5e3101f4501b84fe6eba9697b19/users/5da6fa5f17b741ff86ca357cb5a17d7a/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.372069 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f990dcf-0e9c-4654-a7df-a36924432662 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:36.384586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1023/2076] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:36 2022] PUT /identity/v3/projects/fae906fc3738448cbf57c25c264e25ac/users/34a21df0ef8e40258a5626ec71f1ae9e/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:36.393355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498792b-a753-41ee-9a7f-2d502c52343d admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b5f5e3101f4501b84fe6eba9697b19, user_id=5da6fa5f17b741ff86ca357cb5a17d7a, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:36.393482 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f990dcf-0e9c-4654-a7df-a36924432662 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:36.393560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498792b-a753-41ee-9a7f-2d502c52343d admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:36.395327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:INSERT=9,keystone:SELECT=91 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:36.409396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1054/2077] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:36 2022] PUT /identity/v3/projects/71b5f5e3101f4501b84fe6eba9697b19/users/5da6fa5f17b741ff86ca357cb5a17d7a/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:36.410784 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f7faf-fea7-4ec2-baa5-87dfb00803ba None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.411642 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f7faf-fea7-4ec2-baa5-87dfb00803ba None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.412078 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f7faf-fea7-4ec2-baa5-87dfb00803ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.452132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-200f7faf-fea7-4ec2-baa5-87dfb00803ba None None] MFA Rules not processed for user `34a21df0ef8e40258a5626ec71f1ae9e`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:36.453599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-200f7faf-fea7-4ec2-baa5-87dfb00803ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:36.471736 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f990dcf-0e9c-4654-a7df-a36924432662 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:36.536507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1055/2078] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:36.537634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6e3e5-caff-4c54-ba02-934be95925ed None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.538336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6e3e5-caff-4c54-ba02-934be95925ed None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.539086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6e3e5-caff-4c54-ba02-934be95925ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.568178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ea9d3-0afb-4754-a3bd-63b77bdc1cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:36.573782 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ea9d3-0afb-4754-a3bd-63b77bdc1cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.574074 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ea9d3-0afb-4754-a3bd-63b77bdc1cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.574361 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ea9d3-0afb-4754-a3bd-63b77bdc1cc2 admin admin] PATH_INFO: `/v3/projects/57227bfa15a847c1ae1f81c9d3450fd9/users/57c86b6006b14f5c8464785adb8a13d8/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.584059 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-fde6e3e5-caff-4c54-ba02-934be95925ed None None] MFA Rules not processed for user `5da6fa5f17b741ff86ca357cb5a17d7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:36.585754 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fde6e3e5-caff-4c54-ba02-934be95925ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:36.607910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ea9d3-0afb-4754-a3bd-63b77bdc1cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57227bfa15a847c1ae1f81c9d3450fd9, user_id=57c86b6006b14f5c8464785adb8a13d8, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:36.610690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ea9d3-0afb-4754-a3bd-63b77bdc1cc2 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:36.628861 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1024/2079] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:53:36 2022] PUT /identity/v3/projects/57227bfa15a847c1ae1f81c9d3450fd9/users/57c86b6006b14f5c8464785adb8a13d8/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:36.630395 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ea9d3-0afb-4754-a3bd-63b77bdc1cc2 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:36.634379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-613ea9d3-0afb-4754-a3bd-63b77bdc1cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:36.685637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1056/2080] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:36.686927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147aac6-a14e-40e4-9eae-db7676ef20c5 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.687079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147aac6-a14e-40e4-9eae-db7676ef20c5 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.687437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147aac6-a14e-40e4-9eae-db7676ef20c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.724392 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-1147aac6-a14e-40e4-9eae-db7676ef20c5 None None] MFA Rules not processed for user `57c86b6006b14f5c8464785adb8a13d8`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:36.725088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ea9d3-0afb-4754-a3bd-63b77bdc1cc2 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:36.725612 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1147aac6-a14e-40e4-9eae-db7676ef20c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:36.729389 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e2d7c-8e20-4c70-a19c-3368916a22ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:36.730723 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e2d7c-8e20-4c70-a19c-3368916a22ad service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.731397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e2d7c-8e20-4c70-a19c-3368916a22ad service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.731397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e2d7c-8e20-4c70-a19c-3368916a22ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.733193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-211e2d7c-8e20-4c70-a19c-3368916a22ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:36.750124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:36.806629 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:36.821847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e2d7c-8e20-4c70-a19c-3368916a22ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:36.823366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e2d7c-8e20-4c70-a19c-3368916a22ad service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:36.835311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1025/2081] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:36.836378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e2d7c-8e20-4c70-a19c-3368916a22ad service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:36.837934 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-211e2d7c-8e20-4c70-a19c-3368916a22ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:36.839774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1057/2082] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:36.851536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147aac6-a14e-40e4-9eae-db7676ef20c5 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:36.854828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1147aac6-a14e-40e4-9eae-db7676ef20c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:36.886167 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147aac6-a14e-40e4-9eae-db7676ef20c5 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:36.899977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53083c-50ec-4f64-9d81-2fc86b2032ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:36.901196 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53083c-50ec-4f64-9d81-2fc86b2032ed service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.901746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53083c-50ec-4f64-9d81-2fc86b2032ed service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.902259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53083c-50ec-4f64-9d81-2fc86b2032ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:36.927160 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53083c-50ec-4f64-9d81-2fc86b2032ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:36.927160 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c53083c-50ec-4f64-9d81-2fc86b2032ed service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:36.938853 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1058/2083] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:53:36.966865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e2d7c-8e20-4c70-a19c-3368916a22ad service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:36.972652 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea85ad2-43d0-43e0-adc9-ad48d7d50f22 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d596de177e24ca5a254bda6644a07da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:36.973831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea85ad2-43d0-43e0-adc9-ad48d7d50f22 service glance] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:36.974086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea85ad2-43d0-43e0-adc9-ad48d7d50f22 service glance] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:36.974534 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea85ad2-43d0-43e0-adc9-ad48d7d50f22 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:37.081328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea85ad2-43d0-43e0-adc9-ad48d7d50f22 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:37.083593 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea85ad2-43d0-43e0-adc9-ad48d7d50f22 service glance] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:37.096978 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1026/2084] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:46.932641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=25 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:47.090852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=62 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:47.259387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53083c-50ec-4f64-9d81-2fc86b2032ed service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:47.271144 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53083c-50ec-4f64-9d81-2fc86b2032ed service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:47.276654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b65795-bcbf-4a2d-a05a-d1b8eb8a413f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:47.284190 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b65795-bcbf-4a2d-a05a-d1b8eb8a413f service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:47.284656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b65795-bcbf-4a2d-a05a-d1b8eb8a413f service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:47.285106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b65795-bcbf-4a2d-a05a-d1b8eb8a413f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:47.287264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-31b65795-bcbf-4a2d-a05a-d1b8eb8a413f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:47.376642 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b65795-bcbf-4a2d-a05a-d1b8eb8a413f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:47.378131 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b65795-bcbf-4a2d-a05a-d1b8eb8a413f service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:47.391083 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1059/2085] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:53:48.451879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea85ad2-43d0-43e0-adc9-ad48d7d50f22 service glance] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:48.467444 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea85ad2-43d0-43e0-adc9-ad48d7d50f22 service glance] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:48.479325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642ee2c-deee-4642-9e10-ecef8d6ef84e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:48.480292 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642ee2c-deee-4642-9e10-ecef8d6ef84e service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:48.482927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642ee2c-deee-4642-9e10-ecef8d6ef84e service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:48.483256 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642ee2c-deee-4642-9e10-ecef8d6ef84e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:48.696336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642ee2c-deee-4642-9e10-ecef8d6ef84e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:48.698954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642ee2c-deee-4642-9e10-ecef8d6ef84e service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:48.723271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1027/2086] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:57.385458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=28 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:58.711772 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=30 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:58.904052 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b65795-bcbf-4a2d-a05a-d1b8eb8a413f service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:58.990431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b65795-bcbf-4a2d-a05a-d1b8eb8a413f service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:58.999543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e4d2b-790c-44a5-9db5-333c1f12a3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:59.000602 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e4d2b-790c-44a5-9db5-333c1f12a3fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:59.000997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e4d2b-790c-44a5-9db5-333c1f12a3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:59.001428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e4d2b-790c-44a5-9db5-333c1f12a3fa admin admin] PATH_INFO: `/v3/users/57c86b6006b14f5c8464785adb8a13d8` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:59.003030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e4d2b-790c-44a5-9db5-333c1f12a3fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=57c86b6006b14f5c8464785adb8a13d8)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:59.005242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e4d2b-790c-44a5-9db5-333c1f12a3fa admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:59.079466 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b81e4d2b-790c-44a5-9db5-333c1f12a3fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57c86b6006b14f5c8464785adb8a13d8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:59.080743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b81e4d2b-790c-44a5-9db5-333c1f12a3fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57c86b6006b14f5c8464785adb8a13d8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:59.102153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642ee2c-deee-4642-9e10-ecef8d6ef84e service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:59.111252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e642ee2c-deee-4642-9e10-ecef8d6ef84e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:59.112964 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b81e4d2b-790c-44a5-9db5-333c1f12a3fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57c86b6006b14f5c8464785adb8a13d8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:59.129320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b81e4d2b-790c-44a5-9db5-333c1f12a3fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57c86b6006b14f5c8464785adb8a13d8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:59.144035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642ee2c-deee-4642-9e10-ecef8d6ef84e service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:59.147887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b81e4d2b-790c-44a5-9db5-333c1f12a3fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57c86b6006b14f5c8464785adb8a13d8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:59.155689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7403976-e749-47bd-a33e-7d06fa675483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:59.156949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7403976-e749-47bd-a33e-7d06fa675483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:59.157450 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7403976-e749-47bd-a33e-7d06fa675483 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:59.157860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7403976-e749-47bd-a33e-7d06fa675483 admin admin] PATH_INFO: `/v3/users/95800b197dac4da287e86fe2e9e19fb2` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:59.158993 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1060/2087] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:58 2022] DELETE /identity/v3/users/57c86b6006b14f5c8464785adb8a13d8 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:59.161389 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7403976-e749-47bd-a33e-7d06fa675483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95800b197dac4da287e86fe2e9e19fb2)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:59.163968 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022d40c-8c2b-491e-b8aa-583ac6461b05 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:59.165552 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7403976-e749-47bd-a33e-7d06fa675483 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:59.166441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022d40c-8c2b-491e-b8aa-583ac6461b05 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:59.167170 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022d40c-8c2b-491e-b8aa-583ac6461b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:59.203211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-f022d40c-8c2b-491e-b8aa-583ac6461b05 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:59.204509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f022d40c-8c2b-491e-b8aa-583ac6461b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:59.288273 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d7403976-e749-47bd-a33e-7d06fa675483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95800b197dac4da287e86fe2e9e19fb2'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:59.300102 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d7403976-e749-47bd-a33e-7d06fa675483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95800b197dac4da287e86fe2e9e19fb2'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:59.301120 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d7403976-e749-47bd-a33e-7d06fa675483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95800b197dac4da287e86fe2e9e19fb2'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:59.305647 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1061/2088] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:59.314508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d7403976-e749-47bd-a33e-7d06fa675483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95800b197dac4da287e86fe2e9e19fb2'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:59.315282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d7403976-e749-47bd-a33e-7d06fa675483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95800b197dac4da287e86fe2e9e19fb2'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:59.324751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022d40c-8c2b-491e-b8aa-583ac6461b05 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:59.327600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f022d40c-8c2b-491e-b8aa-583ac6461b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:59.363698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1028/2089] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:59 2022] DELETE /identity/v3/users/95800b197dac4da287e86fe2e9e19fb2 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:53:59.368589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6938c28-b7af-43b0-bbfb-c0fc20f178cd None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:59.369130 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6938c28-b7af-43b0-bbfb-c0fc20f178cd None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:59.369224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6938c28-b7af-43b0-bbfb-c0fc20f178cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:59.372435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:59.392323 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-a6938c28-b7af-43b0-bbfb-c0fc20f178cd None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:59.393702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a6938c28-b7af-43b0-bbfb-c0fc20f178cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:59.423543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022d40c-8c2b-491e-b8aa-583ac6461b05 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:59.476798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1029/2090] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:53:59.477673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6938c28-b7af-43b0-bbfb-c0fc20f178cd None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:59.480919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a6938c28-b7af-43b0-bbfb-c0fc20f178cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:59.493921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6938c28-b7af-43b0-bbfb-c0fc20f178cd None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:59.494946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131a176-ec58-4973-bba2-d448146b4a8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:59.495610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131a176-ec58-4973-bba2-d448146b4a8f service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:59.495939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131a176-ec58-4973-bba2-d448146b4a8f service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:59.496221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131a176-ec58-4973-bba2-d448146b4a8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:59.498047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9131a176-ec58-4973-bba2-d448146b4a8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:59.498638 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a6880-f5a5-4c9e-a79a-4fc06e107df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:59.499526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a6880-f5a5-4c9e-a79a-4fc06e107df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:59.499843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a6880-f5a5-4c9e-a79a-4fc06e107df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:59.500122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a6880-f5a5-4c9e-a79a-4fc06e107df2 admin admin] PATH_INFO: `/v3/users/ad7325a851ee482b8ab61b70ffa92bfe` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:59.501440 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a6880-f5a5-4c9e-a79a-4fc06e107df2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad7325a851ee482b8ab61b70ffa92bfe)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:59.503369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a6880-f5a5-4c9e-a79a-4fc06e107df2 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:59.512700 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131a176-ec58-4973-bba2-d448146b4a8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:59.514182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131a176-ec58-4973-bba2-d448146b4a8f service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:59.526790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1062/2091] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:59.527880 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131a176-ec58-4973-bba2-d448146b4a8f service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:59.529957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9131a176-ec58-4973-bba2-d448146b4a8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:59.543604 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131a176-ec58-4973-bba2-d448146b4a8f service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:59.548446 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30088ce5-a37d-4d65-aa5c-141a15317b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:59.549003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30088ce5-a37d-4d65-aa5c-141a15317b0b service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:59.549337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30088ce5-a37d-4d65-aa5c-141a15317b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:59.549605 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30088ce5-a37d-4d65-aa5c-141a15317b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:59.563416 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30088ce5-a37d-4d65-aa5c-141a15317b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:59.565528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30088ce5-a37d-4d65-aa5c-141a15317b0b service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:59.578610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1063/2092] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:59.581263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-310a6880-f5a5-4c9e-a79a-4fc06e107df2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad7325a851ee482b8ab61b70ffa92bfe'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:59.592928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-310a6880-f5a5-4c9e-a79a-4fc06e107df2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad7325a851ee482b8ab61b70ffa92bfe'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:59.593769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-310a6880-f5a5-4c9e-a79a-4fc06e107df2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad7325a851ee482b8ab61b70ffa92bfe'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:59.605649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-310a6880-f5a5-4c9e-a79a-4fc06e107df2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad7325a851ee482b8ab61b70ffa92bfe'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:59.606589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-310a6880-f5a5-4c9e-a79a-4fc06e107df2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad7325a851ee482b8ab61b70ffa92bfe'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:53:59.613262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:53:59.641195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1030/2093] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:59 2022] DELETE /identity/v3/users/ad7325a851ee482b8ab61b70ffa92bfe => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:53:59.645908 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5097a6-8095-44d6-ab18-135584f4672b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:59.646150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5097a6-8095-44d6-ab18-135584f4672b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:59.646471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5097a6-8095-44d6-ab18-135584f4672b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:59.678135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-2c5097a6-8095-44d6-ab18-135584f4672b None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:53:59.680008 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-2c5097a6-8095-44d6-ab18-135584f4672b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:59.760100 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1064/2094] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:53:59.772272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a6880-f5a5-4c9e-a79a-4fc06e107df2 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:59.774463 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-310a6880-f5a5-4c9e-a79a-4fc06e107df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:59.878409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a6880-f5a5-4c9e-a79a-4fc06e107df2 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:59.885281 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5bccf-36e7-40f9-b809-66a594eb9584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:53:59.886331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5bccf-36e7-40f9-b809-66a594eb9584 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:53:59.886798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5bccf-36e7-40f9-b809-66a594eb9584 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:53:59.887253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5bccf-36e7-40f9-b809-66a594eb9584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:53:59.905662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5bccf-36e7-40f9-b809-66a594eb9584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:53:59.907137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5bccf-36e7-40f9-b809-66a594eb9584 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:53:59.921918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1031/2095] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:53:59.973877 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5097a6-8095-44d6-ab18-135584f4672b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:59.975379 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-2c5097a6-8095-44d6-ab18-135584f4672b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:53:59.989855 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5097a6-8095-44d6-ab18-135584f4672b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:53:59.997747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5bccf-36e7-40f9-b809-66a594eb9584 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:53:59.999881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790ab2f-0c7d-4e96-8663-9e137141e9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:00.000672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e0f5bccf-36e7-40f9-b809-66a594eb9584 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:00.000920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790ab2f-0c7d-4e96-8663-9e137141e9bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:00.001295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790ab2f-0c7d-4e96-8663-9e137141e9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:00.001641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790ab2f-0c7d-4e96-8663-9e137141e9bf admin admin] PATH_INFO: `/v3/projects/57227bfa15a847c1ae1f81c9d3450fd9` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:00.003077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790ab2f-0c7d-4e96-8663-9e137141e9bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=57227bfa15a847c1ae1f81c9d3450fd9)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:00.006052 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790ab2f-0c7d-4e96-8663-9e137141e9bf admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:00.028806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5bccf-36e7-40f9-b809-66a594eb9584 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:00.035806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1669b-84f6-4558-b2c5-7fe474ee9a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:00.037545 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1669b-84f6-4558-b2c5-7fe474ee9a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:00.038300 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1669b-84f6-4558-b2c5-7fe474ee9a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:00.038848 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1669b-84f6-4558-b2c5-7fe474ee9a1d admin admin] PATH_INFO: `/v3/users/ce2f24f2a9404f608017dd019fc08fbe` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:00.045300 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1669b-84f6-4558-b2c5-7fe474ee9a1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce2f24f2a9404f608017dd019fc08fbe)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:00.048612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1669b-84f6-4558-b2c5-7fe474ee9a1d admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:00.087896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d790ab2f-0c7d-4e96-8663-9e137141e9bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57227bfa15a847c1ae1f81c9d3450fd9', 'request_id': 'req-d790ab2f-0c7d-4e96-8663-9e137141e9bf'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.095384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d790ab2f-0c7d-4e96-8663-9e137141e9bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57227bfa15a847c1ae1f81c9d3450fd9', 'request_id': 'req-d790ab2f-0c7d-4e96-8663-9e137141e9bf'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.102352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:00.104794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d790ab2f-0c7d-4e96-8663-9e137141e9bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57227bfa15a847c1ae1f81c9d3450fd9', 'request_id': 'req-d790ab2f-0c7d-4e96-8663-9e137141e9bf'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.143775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d790ab2f-0c7d-4e96-8663-9e137141e9bf admin admin] The token cache is being invalidate because project 57227bfa15a847c1ae1f81c9d3450fd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:00.144261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d790ab2f-0c7d-4e96-8663-9e137141e9bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.145918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d790ab2f-0c7d-4e96-8663-9e137141e9bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.155810 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1065/2096] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:53:59 2022] DELETE /identity/v3/projects/57227bfa15a847c1ae1f81c9d3450fd9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:00.161353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-90b1669b-84f6-4558-b2c5-7fe474ee9a1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce2f24f2a9404f608017dd019fc08fbe'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.161858 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934b793-2a00-4c2f-97e5-2bc0ff37c4d1 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:00.162599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934b793-2a00-4c2f-97e5-2bc0ff37c4d1 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:00.163384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934b793-2a00-4c2f-97e5-2bc0ff37c4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:00.174560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-90b1669b-84f6-4558-b2c5-7fe474ee9a1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce2f24f2a9404f608017dd019fc08fbe'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.175593 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-90b1669b-84f6-4558-b2c5-7fe474ee9a1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce2f24f2a9404f608017dd019fc08fbe'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.186317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-90b1669b-84f6-4558-b2c5-7fe474ee9a1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce2f24f2a9404f608017dd019fc08fbe'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.187335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-90b1669b-84f6-4558-b2c5-7fe474ee9a1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce2f24f2a9404f608017dd019fc08fbe'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.188915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-f934b793-2a00-4c2f-97e5-2bc0ff37c4d1 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:00.190231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f934b793-2a00-4c2f-97e5-2bc0ff37c4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:00.248012 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1032/2097] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:53:59 2022] DELETE /identity/v3/users/ce2f24f2a9404f608017dd019fc08fbe => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:00.252314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac146f9-e55e-444c-a18c-7da8b105224e None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:00.252763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac146f9-e55e-444c-a18c-7da8b105224e None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:00.253362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac146f9-e55e-444c-a18c-7da8b105224e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:00.286168 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-cac146f9-e55e-444c-a18c-7da8b105224e None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:00.288305 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-cac146f9-e55e-444c-a18c-7da8b105224e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:00.295099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1066/2098] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:00.296360 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934b793-2a00-4c2f-97e5-2bc0ff37c4d1 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:00.297833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f934b793-2a00-4c2f-97e5-2bc0ff37c4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:00.314228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934b793-2a00-4c2f-97e5-2bc0ff37c4d1 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:00.440830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:00.443321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:00.448000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:00.448562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:00.450061 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf admin admin] PATH_INFO: `/v3/projects/f1a1374137d844b5abb2db2cad1f1851` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:00.452762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1a1374137d844b5abb2db2cad1f1851)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:00.455624 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:00.464113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1033/2099] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:54:00.464113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07026a-cec6-464e-88bc-1cbb7aa84e6d None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:00.464113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07026a-cec6-464e-88bc-1cbb7aa84e6d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:00.464113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07026a-cec6-464e-88bc-1cbb7aa84e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:00.503997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-1e07026a-cec6-464e-88bc-1cbb7aa84e6d None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:00.505405 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1e07026a-cec6-464e-88bc-1cbb7aa84e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:00.521650 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1034/2100] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:00.522895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07026a-cec6-464e-88bc-1cbb7aa84e6d None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:00.526633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1e07026a-cec6-464e-88bc-1cbb7aa84e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:00.567602 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1a1374137d844b5abb2db2cad1f1851', 'request_id': 'req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.579694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1a1374137d844b5abb2db2cad1f1851', 'request_id': 'req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.588257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1a1374137d844b5abb2db2cad1f1851', 'request_id': 'req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.605311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf admin admin] The token cache is being invalidate because project f1a1374137d844b5abb2db2cad1f1851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:00.605613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.606285 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.611802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1067/2101] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:00 2022] DELETE /identity/v3/projects/f1a1374137d844b5abb2db2cad1f1851 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:00.612755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:00.614281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:00.623409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07026a-cec6-464e-88bc-1cbb7aa84e6d None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:00.644939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:00.703593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114ab1d-6823-4e10-8e9b-e5a2c72f0eaf admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:00.709128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb71784-0a41-4ff9-b5ab-48550bc60c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:00.717387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb71784-0a41-4ff9-b5ab-48550bc60c4a admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:00.717679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb71784-0a41-4ff9-b5ab-48550bc60c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:00.717952 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb71784-0a41-4ff9-b5ab-48550bc60c4a admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:00.718724 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb71784-0a41-4ff9-b5ab-48550bc60c4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:54:00.719038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb71784-0a41-4ff9-b5ab-48550bc60c4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:00.720734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb71784-0a41-4ff9-b5ab-48550bc60c4a admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:00.728322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502b86a-822e-4f0d-acb9-d6088d12c84a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:00.729400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502b86a-822e-4f0d-acb9-d6088d12c84a service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:00.729685 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502b86a-822e-4f0d-acb9-d6088d12c84a service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:00.729966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502b86a-822e-4f0d-acb9-d6088d12c84a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:00.732026 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d502b86a-822e-4f0d-acb9-d6088d12c84a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:00.735523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1068/2102] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:54:00.736840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb71784-0a41-4ff9-b5ab-48550bc60c4a admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:00.738603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-3eb71784-0a41-4ff9-b5ab-48550bc60c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:00.752301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502b86a-822e-4f0d-acb9-d6088d12c84a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:00.753679 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502b86a-822e-4f0d-acb9-d6088d12c84a service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:00.755336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb71784-0a41-4ff9-b5ab-48550bc60c4a admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:00.760916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3acb7f-ceee-4ddf-9096-a1c58ada4245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:00.761808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3acb7f-ceee-4ddf-9096-a1c58ada4245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:00.762096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3acb7f-ceee-4ddf-9096-a1c58ada4245 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:00.762379 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3acb7f-ceee-4ddf-9096-a1c58ada4245 admin admin] PATH_INFO: `/v3/users/802d418d83534e7aa5ab9f10a7309663` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:00.763638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3acb7f-ceee-4ddf-9096-a1c58ada4245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=802d418d83534e7aa5ab9f10a7309663)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:00.765005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1035/2103] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:00.765639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3acb7f-ceee-4ddf-9096-a1c58ada4245 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:00.769187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502b86a-822e-4f0d-acb9-d6088d12c84a service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:00.771136 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d502b86a-822e-4f0d-acb9-d6088d12c84a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:00.787919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502b86a-822e-4f0d-acb9-d6088d12c84a service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:00.795651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:00.796588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:00.796858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:00.797325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64 admin admin] PATH_INFO: `/v3/projects/574b27bf902648ebade2384ff1071d57` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:00.798636 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=574b27bf902648ebade2384ff1071d57)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:00.800716 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:00.836224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-5b3acb7f-ceee-4ddf-9096-a1c58ada4245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '802d418d83534e7aa5ab9f10a7309663'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.837517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-5b3acb7f-ceee-4ddf-9096-a1c58ada4245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '802d418d83534e7aa5ab9f10a7309663'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.859092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-5b3acb7f-ceee-4ddf-9096-a1c58ada4245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '802d418d83534e7aa5ab9f10a7309663'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.872185 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-5b3acb7f-ceee-4ddf-9096-a1c58ada4245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '802d418d83534e7aa5ab9f10a7309663'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.876681 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '574b27bf902648ebade2384ff1071d57', 'request_id': 'req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.887860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-5b3acb7f-ceee-4ddf-9096-a1c58ada4245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '802d418d83534e7aa5ab9f10a7309663'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.889846 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '574b27bf902648ebade2384ff1071d57', 'request_id': 'req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.908025 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1069/2104] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:00 2022] DELETE /identity/v3/users/802d418d83534e7aa5ab9f10a7309663 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:00.909177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '574b27bf902648ebade2384ff1071d57', 'request_id': 'req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.909723 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3acb7f-ceee-4ddf-9096-a1c58ada4245 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:00.911771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-5b3acb7f-ceee-4ddf-9096-a1c58ada4245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:00.930412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64 admin admin] The token cache is being invalidate because project 574b27bf902648ebade2384ff1071d57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:00.930794 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.931854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:00.935941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1036/2105] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:00 2022] DELETE /identity/v3/projects/574b27bf902648ebade2384ff1071d57 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:00.939804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:00.942305 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:01.025672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:01.034384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3acb7f-ceee-4ddf-9096-a1c58ada4245 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:01.043750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3010b0b-86b1-4995-b210-b26ae5aeeade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:01.045209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3010b0b-86b1-4995-b210-b26ae5aeeade admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:01.045664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3010b0b-86b1-4995-b210-b26ae5aeeade admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:01.046018 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3010b0b-86b1-4995-b210-b26ae5aeeade admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:01.047042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3010b0b-86b1-4995-b210-b26ae5aeeade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:01.049989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3010b0b-86b1-4995-b210-b26ae5aeeade admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:01.082555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ce4f5-4a7d-4271-b351-e9bc7f6a2f64 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:01.096171 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1070/2106] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:00 2022] POST /identity/v3/projects => generated 365 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:01.100434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3010b0b-86b1-4995-b210-b26ae5aeeade admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:01.108106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e3010b0b-86b1-4995-b210-b26ae5aeeade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:01.160723 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:01.233730 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347a2d1-6c0c-4595-9a9d-35ca5fad53dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:01.234818 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347a2d1-6c0c-4595-9a9d-35ca5fad53dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:01.235411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347a2d1-6c0c-4595-9a9d-35ca5fad53dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:01.236203 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347a2d1-6c0c-4595-9a9d-35ca5fad53dd admin admin] PATH_INFO: `/v3/users/a723f0f227624405ad449c7d8bf6bd3e` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:01.238260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347a2d1-6c0c-4595-9a9d-35ca5fad53dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a723f0f227624405ad449c7d8bf6bd3e)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:01.247949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347a2d1-6c0c-4595-9a9d-35ca5fad53dd admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:01.284028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3010b0b-86b1-4995-b210-b26ae5aeeade admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:01.291206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e1aef-d523-4f7d-ab12-cfbc1b51e145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:01.293376 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e1aef-d523-4f7d-ab12-cfbc1b51e145 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:01.293782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e1aef-d523-4f7d-ab12-cfbc1b51e145 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:01.294076 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e1aef-d523-4f7d-ab12-cfbc1b51e145 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:01.294865 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e1aef-d523-4f7d-ab12-cfbc1b51e145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:01.297080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e1aef-d523-4f7d-ab12-cfbc1b51e145 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:01.328322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1071/2107] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:01 2022] POST /identity/v3/users => generated 283 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:01.329164 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e1aef-d523-4f7d-ab12-cfbc1b51e145 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:01.331088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ed8e1aef-d523-4f7d-ab12-cfbc1b51e145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:01.351952 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e1aef-d523-4f7d-ab12-cfbc1b51e145 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:01.358165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-1347a2d1-6c0c-4595-9a9d-35ca5fad53dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a723f0f227624405ad449c7d8bf6bd3e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.360467 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66d945-f6ab-4083-9d8a-0be4fca23b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:01.361735 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66d945-f6ab-4083-9d8a-0be4fca23b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:01.362023 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66d945-f6ab-4083-9d8a-0be4fca23b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:01.362310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66d945-f6ab-4083-9d8a-0be4fca23b73 admin admin] PATH_INFO: `/v3/projects/c4b5ef2f930f49b2a1653d943bbe96cd` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:01.363914 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66d945-f6ab-4083-9d8a-0be4fca23b73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4b5ef2f930f49b2a1653d943bbe96cd)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:01.366292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66d945-f6ab-4083-9d8a-0be4fca23b73 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:01.369010 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-1347a2d1-6c0c-4595-9a9d-35ca5fad53dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a723f0f227624405ad449c7d8bf6bd3e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.369472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-1347a2d1-6c0c-4595-9a9d-35ca5fad53dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a723f0f227624405ad449c7d8bf6bd3e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.388526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-1347a2d1-6c0c-4595-9a9d-35ca5fad53dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a723f0f227624405ad449c7d8bf6bd3e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.389301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-1347a2d1-6c0c-4595-9a9d-35ca5fad53dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a723f0f227624405ad449c7d8bf6bd3e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.444315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1037/2108] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:00 2022] DELETE /identity/v3/users/a723f0f227624405ad449c7d8bf6bd3e => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:54:01.445803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347a2d1-6c0c-4595-9a9d-35ca5fad53dd admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:01.456539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-6a66d945-f6ab-4083-9d8a-0be4fca23b73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4b5ef2f930f49b2a1653d943bbe96cd', 'request_id': 'req-6a66d945-f6ab-4083-9d8a-0be4fca23b73'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.462463 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1347a2d1-6c0c-4595-9a9d-35ca5fad53dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:01.467786 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-6a66d945-f6ab-4083-9d8a-0be4fca23b73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4b5ef2f930f49b2a1653d943bbe96cd', 'request_id': 'req-6a66d945-f6ab-4083-9d8a-0be4fca23b73'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.477375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-6a66d945-f6ab-4083-9d8a-0be4fca23b73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4b5ef2f930f49b2a1653d943bbe96cd', 'request_id': 'req-6a66d945-f6ab-4083-9d8a-0be4fca23b73'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.500564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-6a66d945-f6ab-4083-9d8a-0be4fca23b73 admin admin] The token cache is being invalidate because project c4b5ef2f930f49b2a1653d943bbe96cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:01.500993 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-6a66d945-f6ab-4083-9d8a-0be4fca23b73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.502076 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-6a66d945-f6ab-4083-9d8a-0be4fca23b73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.507536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1072/2109] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:01 2022] DELETE /identity/v3/projects/c4b5ef2f930f49b2a1653d943bbe96cd => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:01.508728 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66d945-f6ab-4083-9d8a-0be4fca23b73 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:01.511120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6a66d945-f6ab-4083-9d8a-0be4fca23b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:01.632258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347a2d1-6c0c-4595-9a9d-35ca5fad53dd admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:01.645698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654b8bb-2e52-4897-baee-69fa8b45be6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:01.646484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654b8bb-2e52-4897-baee-69fa8b45be6c admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:01.646768 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654b8bb-2e52-4897-baee-69fa8b45be6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:01.647043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654b8bb-2e52-4897-baee-69fa8b45be6c admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:01.648147 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654b8bb-2e52-4897-baee-69fa8b45be6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:01.652775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66d945-f6ab-4083-9d8a-0be4fca23b73 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:01.653594 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654b8bb-2e52-4897-baee-69fa8b45be6c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:01.659066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:01.664304 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b64d6-bf93-4891-906c-baef54e0ac73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:01.667079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b64d6-bf93-4891-906c-baef54e0ac73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:01.667422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b64d6-bf93-4891-906c-baef54e0ac73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:01.667770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b64d6-bf93-4891-906c-baef54e0ac73 admin admin] PATH_INFO: `/v3/users/3c09acbd99fe4a1dacdd2c097c8f75d8` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:01.669443 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b64d6-bf93-4891-906c-baef54e0ac73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c09acbd99fe4a1dacdd2c097c8f75d8)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:01.673259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b64d6-bf93-4891-906c-baef54e0ac73 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:01.697111 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1038/2110] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:01.698057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654b8bb-2e52-4897-baee-69fa8b45be6c admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:01.702820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5654b8bb-2e52-4897-baee-69fa8b45be6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:01.724820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654b8bb-2e52-4897-baee-69fa8b45be6c admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:01.731913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:01.732870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:01.733187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:01.733580 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9 admin admin] PATH_INFO: `/v3/projects/e7b5dae3a3bd4093bb198c39e4960af1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:01.735037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7b5dae3a3bd4093bb198c39e4960af1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:01.738825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:01.782081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:01.836868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7b5dae3a3bd4093bb198c39e4960af1', 'request_id': 'req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.837532 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-5d0b64d6-bf93-4891-906c-baef54e0ac73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c09acbd99fe4a1dacdd2c097c8f75d8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.838335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-5d0b64d6-bf93-4891-906c-baef54e0ac73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c09acbd99fe4a1dacdd2c097c8f75d8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.851281 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7b5dae3a3bd4093bb198c39e4960af1', 'request_id': 'req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.869357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7b5dae3a3bd4093bb198c39e4960af1', 'request_id': 'req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.874482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-5d0b64d6-bf93-4891-906c-baef54e0ac73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c09acbd99fe4a1dacdd2c097c8f75d8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.891931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-5d0b64d6-bf93-4891-906c-baef54e0ac73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c09acbd99fe4a1dacdd2c097c8f75d8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.894792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9 admin admin] The token cache is being invalidate because project e7b5dae3a3bd4093bb198c39e4960af1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:01.895383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.896469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.899881 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1039/2111] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:01 2022] DELETE /identity/v3/projects/e7b5dae3a3bd4093bb198c39e4960af1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:01.901045 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:01.902105 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-5d0b64d6-bf93-4891-906c-baef54e0ac73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c09acbd99fe4a1dacdd2c097c8f75d8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:01.903418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:01.911068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1073/2112] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:01 2022] DELETE /identity/v3/users/3c09acbd99fe4a1dacdd2c097c8f75d8 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:54:01.912812 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb881d0-823a-48d8-bdca-0b0ce64c461e None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:01.913169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb881d0-823a-48d8-bdca-0b0ce64c461e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:01.913529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb881d0-823a-48d8-bdca-0b0ce64c461e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:01.939033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-abb881d0-823a-48d8-bdca-0b0ce64c461e None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:01.940324 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-abb881d0-823a-48d8-bdca-0b0ce64c461e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:02.032597 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01c28e-9c73-4be3-8a21-23a4ec5177b9 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:02.078462 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1074/2113] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:02.079959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a7c94-ef11-4f46-86a0-8409fb85371e None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:02.080251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a7c94-ef11-4f46-86a0-8409fb85371e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:02.080799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a7c94-ef11-4f46-86a0-8409fb85371e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:02.104143 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-397a7c94-ef11-4f46-86a0-8409fb85371e None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:02.105832 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-397a7c94-ef11-4f46-86a0-8409fb85371e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:02.121090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1075/2114] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:02.127818 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a7c94-ef11-4f46-86a0-8409fb85371e None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:02.148320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a7c94-ef11-4f46-86a0-8409fb85371e None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:02.154842 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc9cb2-6ac6-477f-9627-557f1281e69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:02.155566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc9cb2-6ac6-477f-9627-557f1281e69e admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:02.160531 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc9cb2-6ac6-477f-9627-557f1281e69e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:02.161157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c93ab8-35d3-47d9-9648-99a138ef2387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:02.162334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c93ab8-35d3-47d9-9648-99a138ef2387 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:02.162817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c93ab8-35d3-47d9-9648-99a138ef2387 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:02.163345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c93ab8-35d3-47d9-9648-99a138ef2387 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:02.164648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c93ab8-35d3-47d9-9648-99a138ef2387 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:54:02.164889 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc9cb2-6ac6-477f-9627-557f1281e69e admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:02.165366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c93ab8-35d3-47d9-9648-99a138ef2387 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:02.167117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc9cb2-6ac6-477f-9627-557f1281e69e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:02.169664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c93ab8-35d3-47d9-9648-99a138ef2387 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:02.170887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc9cb2-6ac6-477f-9627-557f1281e69e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:02.188357 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1076/2115] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:54:02.189582 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c93ab8-35d3-47d9-9648-99a138ef2387 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:02.191317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c8c93ab8-35d3-47d9-9648-99a138ef2387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:02.207521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c93ab8-35d3-47d9-9648-99a138ef2387 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:02.211083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1040/2116] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:02.212017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc9cb2-6ac6-477f-9627-557f1281e69e admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:02.214539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f807b4-4b22-4278-82f3-e6a54456a45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:02.217239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f807b4-4b22-4278-82f3-e6a54456a45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:02.217557 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f807b4-4b22-4278-82f3-e6a54456a45d admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:02.218181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f807b4-4b22-4278-82f3-e6a54456a45d admin admin] PATH_INFO: `/v3/projects/cdc87125a7f8474f8907abb35ee0c0e8` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:02.227017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc9cb2-6ac6-477f-9627-557f1281e69e admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:02.227810 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f807b4-4b22-4278-82f3-e6a54456a45d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdc87125a7f8474f8907abb35ee0c0e8)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:02.234182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f807b4-4b22-4278-82f3-e6a54456a45d admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:02.234292 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e5d5e-48d2-4635-a110-0e843abcfbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:02.234776 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e5d5e-48d2-4635-a110-0e843abcfbb4 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:02.235165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e5d5e-48d2-4635-a110-0e843abcfbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:02.235658 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e5d5e-48d2-4635-a110-0e843abcfbb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:02.237224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e5d5e-48d2-4635-a110-0e843abcfbb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:02.242166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e5d5e-48d2-4635-a110-0e843abcfbb4 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:02.290938 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1041/2117] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:02 2022] POST /identity/v3/projects => generated 387 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:02.292348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e5d5e-48d2-4635-a110-0e843abcfbb4 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:02.298151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:02.334202 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-43f807b4-4b22-4278-82f3-e6a54456a45d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdc87125a7f8474f8907abb35ee0c0e8', 'request_id': 'req-43f807b4-4b22-4278-82f3-e6a54456a45d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:02.343350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-43f807b4-4b22-4278-82f3-e6a54456a45d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdc87125a7f8474f8907abb35ee0c0e8', 'request_id': 'req-43f807b4-4b22-4278-82f3-e6a54456a45d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:02.350765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-43f807b4-4b22-4278-82f3-e6a54456a45d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdc87125a7f8474f8907abb35ee0c0e8', 'request_id': 'req-43f807b4-4b22-4278-82f3-e6a54456a45d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:02.392289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-43f807b4-4b22-4278-82f3-e6a54456a45d admin admin] The token cache is being invalidate because project cdc87125a7f8474f8907abb35ee0c0e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:02.406458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-43f807b4-4b22-4278-82f3-e6a54456a45d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:02.407528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-43f807b4-4b22-4278-82f3-e6a54456a45d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:02.410809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1077/2118] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:02 2022] DELETE /identity/v3/projects/cdc87125a7f8474f8907abb35ee0c0e8 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:02.412142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f807b4-4b22-4278-82f3-e6a54456a45d admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:02.415547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-43f807b4-4b22-4278-82f3-e6a54456a45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:02.458476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:02.482411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e5d5e-48d2-4635-a110-0e843abcfbb4 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:02.577713 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd7440-6441-43a0-a409-aa74fb2404a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:02.578743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd7440-6441-43a0-a409-aa74fb2404a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:02.579556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd7440-6441-43a0-a409-aa74fb2404a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:02.579556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd7440-6441-43a0-a409-aa74fb2404a7 admin admin] PATH_INFO: `/v3/projects/546d785812474b61a9ad792dc26afcd8/users/fe76dcaa953f40ab890978e5c30bec65/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:02.582014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f807b4-4b22-4278-82f3-e6a54456a45d admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:02.589267 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c353923-5863-4e72-9ea7-3e87851d3681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:02.589962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c353923-5863-4e72-9ea7-3e87851d3681 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:02.590246 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c353923-5863-4e72-9ea7-3e87851d3681 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:02.590526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c353923-5863-4e72-9ea7-3e87851d3681 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:02.591330 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c353923-5863-4e72-9ea7-3e87851d3681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:02.593361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c353923-5863-4e72-9ea7-3e87851d3681 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:02.621118 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd7440-6441-43a0-a409-aa74fb2404a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=546d785812474b61a9ad792dc26afcd8, user_id=fe76dcaa953f40ab890978e5c30bec65, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:02.624293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd7440-6441-43a0-a409-aa74fb2404a7 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:02.626863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1078/2119] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:02 2022] POST /identity/v3/users => generated 294 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:02.628723 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c353923-5863-4e72-9ea7-3e87851d3681 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:02.630914 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6c353923-5863-4e72-9ea7-3e87851d3681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:02.641898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1042/2120] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:02 2022] PUT /identity/v3/projects/546d785812474b61a9ad792dc26afcd8/users/fe76dcaa953f40ab890978e5c30bec65/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:02.643143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd7440-6441-43a0-a409-aa74fb2404a7 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:02.660938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c353923-5863-4e72-9ea7-3e87851d3681 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:02.725095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd7440-6441-43a0-a409-aa74fb2404a7 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:02.732590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fef0b-b879-4291-8846-130271ce6c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:02.733285 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fef0b-b879-4291-8846-130271ce6c60 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:02.733573 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fef0b-b879-4291-8846-130271ce6c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:02.733851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fef0b-b879-4291-8846-130271ce6c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:02.734591 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815fef0b-b879-4291-8846-130271ce6c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:02.735516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb7744-e32b-4625-9de0-41a3884a607e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:02.736512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815fef0b-b879-4291-8846-130271ce6c60 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:02.736734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb7744-e32b-4625-9de0-41a3884a607e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:02.737169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb7744-e32b-4625-9de0-41a3884a607e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:02.737453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb7744-e32b-4625-9de0-41a3884a607e admin admin] PATH_INFO: `/v3/users/34a21df0ef8e40258a5626ec71f1ae9e` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:02.738871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb7744-e32b-4625-9de0-41a3884a607e admin admin] RBAC: Authorizing `identity:delete_user(user_id=34a21df0ef8e40258a5626ec71f1ae9e)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:02.740855 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb7744-e32b-4625-9de0-41a3884a607e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:02.751884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1043/2121] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:02.751884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fef0b-b879-4291-8846-130271ce6c60 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:02.751884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-815fef0b-b879-4291-8846-130271ce6c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:02.765204 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fef0b-b879-4291-8846-130271ce6c60 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:02.770185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397692c-3cb8-4aa0-9a31-3a308b174983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:02.771294 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397692c-3cb8-4aa0-9a31-3a308b174983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:02.771452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397692c-3cb8-4aa0-9a31-3a308b174983 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:02.771874 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397692c-3cb8-4aa0-9a31-3a308b174983 admin admin] PATH_INFO: `/v3/projects/aea4d2b8b59d46ed8f0c1c0ce42b6341` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:02.773471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397692c-3cb8-4aa0-9a31-3a308b174983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aea4d2b8b59d46ed8f0c1c0ce42b6341)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:02.776009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397692c-3cb8-4aa0-9a31-3a308b174983 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:02.800258 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-20eb7744-e32b-4625-9de0-41a3884a607e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34a21df0ef8e40258a5626ec71f1ae9e'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:02.801246 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-20eb7744-e32b-4625-9de0-41a3884a607e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34a21df0ef8e40258a5626ec71f1ae9e'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:02.805585 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:02.818686 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-20eb7744-e32b-4625-9de0-41a3884a607e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34a21df0ef8e40258a5626ec71f1ae9e'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:02.828549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-20eb7744-e32b-4625-9de0-41a3884a607e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34a21df0ef8e40258a5626ec71f1ae9e'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:02.836524 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a397692c-3cb8-4aa0-9a31-3a308b174983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aea4d2b8b59d46ed8f0c1c0ce42b6341', 'request_id': 'req-a397692c-3cb8-4aa0-9a31-3a308b174983'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:02.837673 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-20eb7744-e32b-4625-9de0-41a3884a607e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34a21df0ef8e40258a5626ec71f1ae9e'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:02.848874 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a397692c-3cb8-4aa0-9a31-3a308b174983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aea4d2b8b59d46ed8f0c1c0ce42b6341', 'request_id': 'req-a397692c-3cb8-4aa0-9a31-3a308b174983'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:02.849460 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1079/2122] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:02 2022] DELETE /identity/v3/users/34a21df0ef8e40258a5626ec71f1ae9e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:02.850548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a268ef3-e706-4fe6-9f5b-263eff4dfa98 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:02.850846 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a268ef3-e706-4fe6-9f5b-263eff4dfa98 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:02.851904 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a268ef3-e706-4fe6-9f5b-263eff4dfa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:02.863435 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a397692c-3cb8-4aa0-9a31-3a308b174983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aea4d2b8b59d46ed8f0c1c0ce42b6341', 'request_id': 'req-a397692c-3cb8-4aa0-9a31-3a308b174983'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:02.884162 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a397692c-3cb8-4aa0-9a31-3a308b174983 admin admin] The token cache is being invalidate because project aea4d2b8b59d46ed8f0c1c0ce42b6341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:02.885090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a397692c-3cb8-4aa0-9a31-3a308b174983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:02.885826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a397692c-3cb8-4aa0-9a31-3a308b174983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:02.888530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-3a268ef3-e706-4fe6-9f5b-263eff4dfa98 None None] MFA Rules not processed for user `fe76dcaa953f40ab890978e5c30bec65`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:02.890224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-3a268ef3-e706-4fe6-9f5b-263eff4dfa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:02.891613 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1044/2123] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:02 2022] DELETE /identity/v3/projects/aea4d2b8b59d46ed8f0c1c0ce42b6341 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:02.892708 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397692c-3cb8-4aa0-9a31-3a308b174983 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:02.894463 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a397692c-3cb8-4aa0-9a31-3a308b174983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:02.895934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:02.997304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397692c-3cb8-4aa0-9a31-3a308b174983 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:03.011968 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1080/2124] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:03.014755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9fc50-d582-4234-b9b4-965ecbaad4a5 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:03.015129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9fc50-d582-4234-b9b4-965ecbaad4a5 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:03.015815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9fc50-d582-4234-b9b4-965ecbaad4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:03.051078 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d3e9fc50-d582-4234-b9b4-965ecbaad4a5 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:03.052410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d3e9fc50-d582-4234-b9b4-965ecbaad4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:03.100045 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6384f-4eae-4d2e-b8d8-563ee409c47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:03.102315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6384f-4eae-4d2e-b8d8-563ee409c47c admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:03.107021 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6384f-4eae-4d2e-b8d8-563ee409c47c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:03.107878 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6384f-4eae-4d2e-b8d8-563ee409c47c admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:03.109163 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6384f-4eae-4d2e-b8d8-563ee409c47c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:03.111871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6384f-4eae-4d2e-b8d8-563ee409c47c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:03.135287 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1045/2125] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:03.136412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6384f-4eae-4d2e-b8d8-563ee409c47c admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:03.138837 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-41d6384f-4eae-4d2e-b8d8-563ee409c47c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:03.175752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1081/2126] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:03.177269 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9fc50-d582-4234-b9b4-965ecbaad4a5 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:03.189419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9fc50-d582-4234-b9b4-965ecbaad4a5 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:03.196077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120923e-d502-4fb4-ad03-f5ede78a169a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:03.198119 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120923e-d502-4fb4-ad03-f5ede78a169a admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:03.198629 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120923e-d502-4fb4-ad03-f5ede78a169a admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:03.199145 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120923e-d502-4fb4-ad03-f5ede78a169a admin admin] PATH_INFO: `/v3/projects/28bdd0aa542c4816bbcd114b520e1129/users/1f31fbf48d684f7b9014d6db0961a9d8/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:03.220637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120923e-d502-4fb4-ad03-f5ede78a169a admin admin] RBAC: Authorizing `identity:create_grant(project_id=28bdd0aa542c4816bbcd114b520e1129, user_id=1f31fbf48d684f7b9014d6db0961a9d8, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:03.223648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120923e-d502-4fb4-ad03-f5ede78a169a admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:03.236702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6384f-4eae-4d2e-b8d8-563ee409c47c admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:03.250391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1082/2127] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:03 2022] PUT /identity/v3/projects/28bdd0aa542c4816bbcd114b520e1129/users/1f31fbf48d684f7b9014d6db0961a9d8/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:03.250391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a072b23-4d70-4bb3-ad57-8d48ab205ad0 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:03.250391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a072b23-4d70-4bb3-ad57-8d48ab205ad0 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:03.250391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a072b23-4d70-4bb3-ad57-8d48ab205ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:03.254349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:03.295270 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-2a072b23-4d70-4bb3-ad57-8d48ab205ad0 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:03.297043 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-2a072b23-4d70-4bb3-ad57-8d48ab205ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:03.357465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d1392-6afb-41e0-ad6c-44a0b4897480 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:03.358613 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d1392-6afb-41e0-ad6c-44a0b4897480 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:03.358983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d1392-6afb-41e0-ad6c-44a0b4897480 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:03.359357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d1392-6afb-41e0-ad6c-44a0b4897480 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:03.412684 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1083/2128] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:03.419099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e5071-0078-4d21-8752-92d0fa2213c0 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:03.419804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e5071-0078-4d21-8752-92d0fa2213c0 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:03.419916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e5071-0078-4d21-8752-92d0fa2213c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:03.448842 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d1392-6afb-41e0-ad6c-44a0b4897480 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:03.450422 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d1392-6afb-41e0-ad6c-44a0b4897480 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:03.461815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1046/2129] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:03.462295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d33e5071-0078-4d21-8752-92d0fa2213c0 None None] MFA Rules not processed for user `1f31fbf48d684f7b9014d6db0961a9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:03.464035 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d33e5071-0078-4d21-8752-92d0fa2213c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:03.473909 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d1392-6afb-41e0-ad6c-44a0b4897480 service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:03.479434 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-570d1392-6afb-41e0-ad6c-44a0b4897480 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:03.493061 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:03.505250 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d1392-6afb-41e0-ad6c-44a0b4897480 service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:03.512033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:03.513113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:03.513323 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:03.513610 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8 admin admin] PATH_INFO: `/v3/projects/fae906fc3738448cbf57c25c264e25ac` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:03.514913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fae906fc3738448cbf57c25c264e25ac)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:03.518316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:03.591312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1084/2130] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:03.592516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e5071-0078-4d21-8752-92d0fa2213c0 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:03.617298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e5071-0078-4d21-8752-92d0fa2213c0 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:03.621122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fae906fc3738448cbf57c25c264e25ac', 'request_id': 'req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:03.652302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fae906fc3738448cbf57c25c264e25ac', 'request_id': 'req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:03.664217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fae906fc3738448cbf57c25c264e25ac', 'request_id': 'req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:03.677307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8 admin admin] The token cache is being invalidate because project fae906fc3738448cbf57c25c264e25ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:03.677583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:03.681290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:03.682949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1047/2131] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:03 2022] DELETE /identity/v3/projects/fae906fc3738448cbf57c25c264e25ac => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:03.683867 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:03.685260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:03.733129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea1f35-8f92-4ae2-a1ba-dcf0a076b6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:03.733824 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea1f35-8f92-4ae2-a1ba-dcf0a076b6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:03.734525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea1f35-8f92-4ae2-a1ba-dcf0a076b6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:03.734872 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea1f35-8f92-4ae2-a1ba-dcf0a076b6e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:03.735606 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea1f35-8f92-4ae2-a1ba-dcf0a076b6e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:54:03.736201 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea1f35-8f92-4ae2-a1ba-dcf0a076b6e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:03.738814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea1f35-8f92-4ae2-a1ba-dcf0a076b6e7 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:03.753280 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1085/2132] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:54:03.754662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83946366-ec1a-469d-bc21-b7b7afc891e7 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:03.754954 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83946366-ec1a-469d-bc21-b7b7afc891e7 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:03.755225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83946366-ec1a-469d-bc21-b7b7afc891e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:03.757369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:03.778094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-83946366-ec1a-469d-bc21-b7b7afc891e7 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:03.779406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-83946366-ec1a-469d-bc21-b7b7afc891e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:03.785329 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1d987-3c57-44be-8c77-86e6d3d2e6e8 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:03.793983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1086/2133] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:03.793983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83946366-ec1a-469d-bc21-b7b7afc891e7 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:03.793983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-83946366-ec1a-469d-bc21-b7b7afc891e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:03.796766 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab978cee-2b74-4553-b98e-de342458d5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:03.797550 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab978cee-2b74-4553-b98e-de342458d5a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:03.797903 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab978cee-2b74-4553-b98e-de342458d5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:03.798319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab978cee-2b74-4553-b98e-de342458d5a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:03.799037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab978cee-2b74-4553-b98e-de342458d5a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:03.801123 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab978cee-2b74-4553-b98e-de342458d5a9 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:03.803385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83946366-ec1a-469d-bc21-b7b7afc891e7 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:03.809507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cc5bf-d1f4-4afc-8b6c-6dff1c1ecd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:03.810173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cc5bf-d1f4-4afc-8b6c-6dff1c1ecd2b admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:03.810447 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cc5bf-d1f4-4afc-8b6c-6dff1c1ecd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:03.810714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cc5bf-d1f4-4afc-8b6c-6dff1c1ecd2b admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:03.811439 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cc5bf-d1f4-4afc-8b6c-6dff1c1ecd2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:03.813462 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cc5bf-d1f4-4afc-8b6c-6dff1c1ecd2b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:03.840750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1048/2134] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:03 2022] POST /identity/v3/projects => generated 387 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:03.842390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da644f4-d3cb-4b9f-ad3f-4f18bd5e7ec1 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:03.842845 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da644f4-d3cb-4b9f-ad3f-4f18bd5e7ec1 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:03.843133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da644f4-d3cb-4b9f-ad3f-4f18bd5e7ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:03.845848 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1087/2135] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:03 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:03.847195 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cc5bf-d1f4-4afc-8b6c-6dff1c1ecd2b admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:03.881112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-1da644f4-d3cb-4b9f-ad3f-4f18bd5e7ec1 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:03.883091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1da644f4-d3cb-4b9f-ad3f-4f18bd5e7ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:03.954614 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cc5bf-d1f4-4afc-8b6c-6dff1c1ecd2b admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:03.961191 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3947c-4fd1-4899-8112-c4b7c782df90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:03.961708 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3947c-4fd1-4899-8112-c4b7c782df90 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:03.962450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3947c-4fd1-4899-8112-c4b7c782df90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:03.962740 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3947c-4fd1-4899-8112-c4b7c782df90 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:03.963543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3947c-4fd1-4899-8112-c4b7c782df90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:03.965435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3947c-4fd1-4899-8112-c4b7c782df90 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:03.982974 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1049/2136] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:03.983988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da644f4-d3cb-4b9f-ad3f-4f18bd5e7ec1 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:03.994935 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da644f4-d3cb-4b9f-ad3f-4f18bd5e7ec1 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:04.001896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1088/2137] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:03 2022] POST /identity/v3/users => generated 300 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:04.001896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3947c-4fd1-4899-8112-c4b7c782df90 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:04.003623 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae5164-93bf-4938-b30d-a2e19e6185ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:04.004452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae5164-93bf-4938-b30d-a2e19e6185ec admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.004682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae5164-93bf-4938-b30d-a2e19e6185ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.004895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae5164-93bf-4938-b30d-a2e19e6185ec admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.005598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae5164-93bf-4938-b30d-a2e19e6185ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:04.008279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae5164-93bf-4938-b30d-a2e19e6185ec admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:04.011443 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3947c-4fd1-4899-8112-c4b7c782df90 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:04.018181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9edf5d4-a22f-4393-bdf8-9379b4e4e626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:04.019023 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9edf5d4-a22f-4393-bdf8-9379b4e4e626 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.019863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9edf5d4-a22f-4393-bdf8-9379b4e4e626 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.019863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9edf5d4-a22f-4393-bdf8-9379b4e4e626 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.020961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9edf5d4-a22f-4393-bdf8-9379b4e4e626 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:54:04.021409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9edf5d4-a22f-4393-bdf8-9379b4e4e626 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:04.023883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9edf5d4-a22f-4393-bdf8-9379b4e4e626 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:04.025930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:04.035834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1050/2138] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:03 2022] POST /identity/v3/users => generated 295 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:04.035834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae5164-93bf-4938-b30d-a2e19e6185ec admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:04.045221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1089/2139] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:54:04.049472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9edf5d4-a22f-4393-bdf8-9379b4e4e626 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:04.049579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae5164-93bf-4938-b30d-a2e19e6185ec admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:04.053366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fbc54-d414-44c2-a87e-74b326024b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:04.054127 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fbc54-d414-44c2-a87e-74b326024b45 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.054400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fbc54-d414-44c2-a87e-74b326024b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.054695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fbc54-d414-44c2-a87e-74b326024b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.055598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fbc54-d414-44c2-a87e-74b326024b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:04.057249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fbc54-d414-44c2-a87e-74b326024b45 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:04.061006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9edf5d4-a22f-4393-bdf8-9379b4e4e626 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:04.066209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d94c7-43b2-429e-a495-dd1625ff069e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:04.066801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d94c7-43b2-429e-a495-dd1625ff069e admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.067244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d94c7-43b2-429e-a495-dd1625ff069e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.067522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d94c7-43b2-429e-a495-dd1625ff069e admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.068133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1051/2140] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:04.068333 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d94c7-43b2-429e-a495-dd1625ff069e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:04.069038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fbc54-d414-44c2-a87e-74b326024b45 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:04.070605 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d94c7-43b2-429e-a495-dd1625ff069e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:04.082073 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fbc54-d414-44c2-a87e-74b326024b45 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:04.087335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1090/2141] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:04.087900 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640b59b-e93a-40c0-ad1e-5ddc2ecf8c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:04.088217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d94c7-43b2-429e-a495-dd1625ff069e admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:04.088830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640b59b-e93a-40c0-ad1e-5ddc2ecf8c99 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.089188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640b59b-e93a-40c0-ad1e-5ddc2ecf8c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.089554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640b59b-e93a-40c0-ad1e-5ddc2ecf8c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.090699 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640b59b-e93a-40c0-ad1e-5ddc2ecf8c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:04.092821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640b59b-e93a-40c0-ad1e-5ddc2ecf8c99 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:04.099290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d94c7-43b2-429e-a495-dd1625ff069e admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:04.104433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010a015-fde6-470b-907f-ed0cd702270c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:04.105324 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010a015-fde6-470b-907f-ed0cd702270c admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.105607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010a015-fde6-470b-907f-ed0cd702270c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.105883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010a015-fde6-470b-907f-ed0cd702270c admin admin] PATH_INFO: `/v3/projects/93e661cb4a6d4d04a091c528998d06ee/users/7f8b7c8ae31347119a712d4e82a0e30c/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.127167 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010a015-fde6-470b-907f-ed0cd702270c admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e661cb4a6d4d04a091c528998d06ee, user_id=7f8b7c8ae31347119a712d4e82a0e30c, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:04.129215 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1052/2142] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:04 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:04.130619 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640b59b-e93a-40c0-ad1e-5ddc2ecf8c99 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:04.134753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010a015-fde6-470b-907f-ed0cd702270c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:04.151871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1091/2143] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:04 2022] PUT /identity/v3/projects/93e661cb4a6d4d04a091c528998d06ee/users/7f8b7c8ae31347119a712d4e82a0e30c/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:04.155325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010a015-fde6-470b-907f-ed0cd702270c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:04.238787 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640b59b-e93a-40c0-ad1e-5ddc2ecf8c99 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:04.258767 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010a015-fde6-470b-907f-ed0cd702270c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:04.264663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21915f4d-d3da-43dc-b554-f3d34897cd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:04.265503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21915f4d-d3da-43dc-b554-f3d34897cd8c admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.265792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21915f4d-d3da-43dc-b554-f3d34897cd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.266067 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21915f4d-d3da-43dc-b554-f3d34897cd8c admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.266769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21915f4d-d3da-43dc-b554-f3d34897cd8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:04.269331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21915f4d-d3da-43dc-b554-f3d34897cd8c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:04.280173 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:04.306517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1092/2144] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:04 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:04.307540 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21915f4d-d3da-43dc-b554-f3d34897cd8c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:04.322036 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21915f4d-d3da-43dc-b554-f3d34897cd8c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:04.324286 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009190cc-af15-4c05-a1b0-1f0cea051084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:04.325094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009190cc-af15-4c05-a1b0-1f0cea051084 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.325541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009190cc-af15-4c05-a1b0-1f0cea051084 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.326029 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009190cc-af15-4c05-a1b0-1f0cea051084 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.326967 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009190cc-af15-4c05-a1b0-1f0cea051084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:04.328370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec392f2-1770-458d-bd58-63d473942d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:04.328600 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009190cc-af15-4c05-a1b0-1f0cea051084 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:04.329055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec392f2-1770-458d-bd58-63d473942d43 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.329339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec392f2-1770-458d-bd58-63d473942d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.329801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec392f2-1770-458d-bd58-63d473942d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.330628 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec392f2-1770-458d-bd58-63d473942d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:04.332177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec392f2-1770-458d-bd58-63d473942d43 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:04.349547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1093/2145] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:04.351222 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec392f2-1770-458d-bd58-63d473942d43 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:04.351506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1053/2146] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:04.353738 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009190cc-af15-4c05-a1b0-1f0cea051084 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:04.367653 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec392f2-1770-458d-bd58-63d473942d43 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:04.370398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009190cc-af15-4c05-a1b0-1f0cea051084 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:04.373957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c3b3a-be04-458b-9d73-7bd9c3e332a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:04.374236 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c3b3a-be04-458b-9d73-7bd9c3e332a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.374592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c3b3a-be04-458b-9d73-7bd9c3e332a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.374936 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c3b3a-be04-458b-9d73-7bd9c3e332a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.375329 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75486a-2a86-4e3c-b5a3-9f575484b20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:04.375884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c3b3a-be04-458b-9d73-7bd9c3e332a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:04.376343 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75486a-2a86-4e3c-b5a3-9f575484b20f admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.376631 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75486a-2a86-4e3c-b5a3-9f575484b20f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.376909 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75486a-2a86-4e3c-b5a3-9f575484b20f admin admin] PATH_INFO: `/v3/projects/93e661cb4a6d4d04a091c528998d06ee/users/7f8b7c8ae31347119a712d4e82a0e30c/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.378125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c3b3a-be04-458b-9d73-7bd9c3e332a8 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:04.379195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75486a-2a86-4e3c-b5a3-9f575484b20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e661cb4a6d4d04a091c528998d06ee, user_id=7f8b7c8ae31347119a712d4e82a0e30c, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:04.382929 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75486a-2a86-4e3c-b5a3-9f575484b20f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:04.396098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1054/2147] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:04 2022] PUT /identity/v3/projects/93e661cb4a6d4d04a091c528998d06ee/users/7f8b7c8ae31347119a712d4e82a0e30c/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:54:04.405422 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75486a-2a86-4e3c-b5a3-9f575484b20f admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:04.405553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1094/2148] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:04.405553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c3b3a-be04-458b-9d73-7bd9c3e332a8 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:04.414947 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c3b3a-be04-458b-9d73-7bd9c3e332a8 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:04.488814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d4981-e1e6-47f1-a0b7-546f712dd144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:04.489530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d4981-e1e6-47f1-a0b7-546f712dd144 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.489822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d4981-e1e6-47f1-a0b7-546f712dd144 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.490432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d4981-e1e6-47f1-a0b7-546f712dd144 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.491351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d4981-e1e6-47f1-a0b7-546f712dd144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:04.494171 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d4981-e1e6-47f1-a0b7-546f712dd144 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:04.511596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1095/2149] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:54:04.512492 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75486a-2a86-4e3c-b5a3-9f575484b20f admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:04.513383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0095537-bd57-4735-a122-f7d07f67535b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.513684 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0095537-bd57-4735-a122-f7d07f67535b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.514369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0095537-bd57-4735-a122-f7d07f67535b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.520302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd39550-cd50-432b-81da-d1c59215a80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:04.521310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd39550-cd50-432b-81da-d1c59215a80e admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.521592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd39550-cd50-432b-81da-d1c59215a80e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.521866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd39550-cd50-432b-81da-d1c59215a80e admin admin] PATH_INFO: `/v3/projects/04ec27237ddc47b09a744d525e107c85/users/b91baa8b29674a46a9a40f2c8f5de355/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.541367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd39550-cd50-432b-81da-d1c59215a80e admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ec27237ddc47b09a744d525e107c85, user_id=b91baa8b29674a46a9a40f2c8f5de355, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:04.544206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd39550-cd50-432b-81da-d1c59215a80e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:04.565055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-b0095537-bd57-4735-a122-f7d07f67535b None None] MFA Rules not processed for user `7f8b7c8ae31347119a712d4e82a0e30c`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:04.567436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b0095537-bd57-4735-a122-f7d07f67535b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:04.569405 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1055/2150] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:04 2022] PUT /identity/v3/projects/04ec27237ddc47b09a744d525e107c85/users/b91baa8b29674a46a9a40f2c8f5de355/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:04.571199 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd39550-cd50-432b-81da-d1c59215a80e admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:04.584445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:04.673011 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd39550-cd50-432b-81da-d1c59215a80e admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:04.681752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378c8d6-ae81-4cb2-93f3-30e5253059e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:04.682675 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1096/2151] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:04.682786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378c8d6-ae81-4cb2-93f3-30e5253059e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.682935 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378c8d6-ae81-4cb2-93f3-30e5253059e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.683218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378c8d6-ae81-4cb2-93f3-30e5253059e7 admin admin] PATH_INFO: `/v3/projects/ca3d3b69646549fdbdaf9377921bd610/users/26f0f6230a6b4f54b43e16ae12aa7a2e/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.687402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94494408-c15a-44ff-9523-7edb9d726e80 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.687402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94494408-c15a-44ff-9523-7edb9d726e80 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.687402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94494408-c15a-44ff-9523-7edb9d726e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.709999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378c8d6-ae81-4cb2-93f3-30e5253059e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca3d3b69646549fdbdaf9377921bd610, user_id=26f0f6230a6b4f54b43e16ae12aa7a2e, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:04.712991 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378c8d6-ae81-4cb2-93f3-30e5253059e7 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:04.725177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-94494408-c15a-44ff-9523-7edb9d726e80 None None] MFA Rules not processed for user `b91baa8b29674a46a9a40f2c8f5de355`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:04.727261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-94494408-c15a-44ff-9523-7edb9d726e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:04.731927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1056/2152] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:04 2022] PUT /identity/v3/projects/ca3d3b69646549fdbdaf9377921bd610/users/26f0f6230a6b4f54b43e16ae12aa7a2e/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:04.733360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378c8d6-ae81-4cb2-93f3-30e5253059e7 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:04.744443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1378c8d6-ae81-4cb2-93f3-30e5253059e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:04.768034 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:04.819277 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1097/2153] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:04.820845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc2598-3281-4d4b-8565-cdece6b13b6c None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.821351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc2598-3281-4d4b-8565-cdece6b13b6c None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.821566 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc2598-3281-4d4b-8565-cdece6b13b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.827797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378c8d6-ae81-4cb2-93f3-30e5253059e7 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:04.831876 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0314622-bd0b-434f-a700-77facdd3942e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:04.832863 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0314622-bd0b-434f-a700-77facdd3942e service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:04.833329 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0314622-bd0b-434f-a700-77facdd3942e service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:04.833782 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0314622-bd0b-434f-a700-77facdd3942e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:04.860661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e9fc2598-3281-4d4b-8565-cdece6b13b6c None None] MFA Rules not processed for user `26f0f6230a6b4f54b43e16ae12aa7a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:04.862525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e9fc2598-3281-4d4b-8565-cdece6b13b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:04.906394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0314622-bd0b-434f-a700-77facdd3942e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:04.907900 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0314622-bd0b-434f-a700-77facdd3942e service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:04.920006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1057/2154] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:04.920667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0314622-bd0b-434f-a700-77facdd3942e service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:04.962985 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1098/2155] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:04.968781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc2598-3281-4d4b-8565-cdece6b13b6c None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:05.008745 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:05.016240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0314622-bd0b-434f-a700-77facdd3942e service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:05.029486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157dc87-04de-46aa-9e6e-56001d76155d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:05.030510 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157dc87-04de-46aa-9e6e-56001d76155d admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:05.030795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157dc87-04de-46aa-9e6e-56001d76155d admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:05.031237 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157dc87-04de-46aa-9e6e-56001d76155d admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:05.032281 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157dc87-04de-46aa-9e6e-56001d76155d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:05.035067 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157dc87-04de-46aa-9e6e-56001d76155d admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:05.055581 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc2598-3281-4d4b-8565-cdece6b13b6c None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:05.071270 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5b94c-a9d8-451d-8283-3db38620137a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:05.071986 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5b94c-a9d8-451d-8283-3db38620137a admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:05.072414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5b94c-a9d8-451d-8283-3db38620137a admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:05.072578 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5b94c-a9d8-451d-8283-3db38620137a admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:05.073227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5b94c-a9d8-451d-8283-3db38620137a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:05.075200 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5b94c-a9d8-451d-8283-3db38620137a admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:05.081038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1058/2156] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:04 2022] POST /identity/v3/projects => generated 391 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:05.081038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157dc87-04de-46aa-9e6e-56001d76155d admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:05.122405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1099/2157] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:04 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:54:05.122405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5b94c-a9d8-451d-8283-3db38620137a admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:05.185532 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:05.195761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157dc87-04de-46aa-9e6e-56001d76155d admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:05.221779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5b94c-a9d8-451d-8283-3db38620137a admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:05.234226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110c305-7d7a-4126-a1a0-226ddc409aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:05.234226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110c305-7d7a-4126-a1a0-226ddc409aba admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:05.234226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110c305-7d7a-4126-a1a0-226ddc409aba admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:05.234226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110c305-7d7a-4126-a1a0-226ddc409aba admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:05.234888 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110c305-7d7a-4126-a1a0-226ddc409aba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:05.237667 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110c305-7d7a-4126-a1a0-226ddc409aba admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:05.260885 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1100/2158] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:05 2022] POST /identity/v3/users => generated 302 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:05.264985 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110c305-7d7a-4126-a1a0-226ddc409aba admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:05.278078 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110c305-7d7a-4126-a1a0-226ddc409aba admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:05.285629 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9db37-d324-4b03-bb91-4875ef1c4388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:05.285793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8013c-856d-4e6f-828c-a323435d1f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:05.286655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8013c-856d-4e6f-828c-a323435d1f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:05.286756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9db37-d324-4b03-bb91-4875ef1c4388 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:05.286937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8013c-856d-4e6f-828c-a323435d1f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:05.287195 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9db37-d324-4b03-bb91-4875ef1c4388 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:05.287394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8013c-856d-4e6f-828c-a323435d1f16 admin admin] PATH_INFO: `/v3/users/1f31fbf48d684f7b9014d6db0961a9d8` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:05.287671 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9db37-d324-4b03-bb91-4875ef1c4388 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:05.288562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9db37-d324-4b03-bb91-4875ef1c4388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:05.290650 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8013c-856d-4e6f-828c-a323435d1f16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f31fbf48d684f7b9014d6db0961a9d8)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:05.293007 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8013c-856d-4e6f-828c-a323435d1f16 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:05.293206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9db37-d324-4b03-bb91-4875ef1c4388 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:05.324235 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1101/2159] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:05 2022] POST /identity/v3/users => generated 293 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:05.324235 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9db37-d324-4b03-bb91-4875ef1c4388 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:05.337203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9db37-d324-4b03-bb91-4875ef1c4388 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:05.349058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6bd1a-9fe7-447f-b7ab-c38a058d21c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:05.349811 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6bd1a-9fe7-447f-b7ab-c38a058d21c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:05.350411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6bd1a-9fe7-447f-b7ab-c38a058d21c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:05.350411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6bd1a-9fe7-447f-b7ab-c38a058d21c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:05.355542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6bd1a-9fe7-447f-b7ab-c38a058d21c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:05.357249 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6bd1a-9fe7-447f-b7ab-c38a058d21c7 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:05.376138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1102/2160] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:05.377686 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6bd1a-9fe7-447f-b7ab-c38a058d21c7 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:05.385342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-bee8013c-856d-4e6f-828c-a323435d1f16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f31fbf48d684f7b9014d6db0961a9d8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:05.390677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6bd1a-9fe7-447f-b7ab-c38a058d21c7 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:05.397708 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c9d3b-aecb-4be6-bf56-7c8ed6d928fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:05.399259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c9d3b-aecb-4be6-bf56-7c8ed6d928fe admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:05.399516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c9d3b-aecb-4be6-bf56-7c8ed6d928fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:05.399983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c9d3b-aecb-4be6-bf56-7c8ed6d928fe admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:05.401080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c9d3b-aecb-4be6-bf56-7c8ed6d928fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:05.401970 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-bee8013c-856d-4e6f-828c-a323435d1f16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f31fbf48d684f7b9014d6db0961a9d8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:05.404023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-bee8013c-856d-4e6f-828c-a323435d1f16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f31fbf48d684f7b9014d6db0961a9d8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:05.405956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c9d3b-aecb-4be6-bf56-7c8ed6d928fe admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:05.421047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1103/2161] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:05.425187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c9d3b-aecb-4be6-bf56-7c8ed6d928fe admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:05.432279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ec1c9d3b-aecb-4be6-bf56-7c8ed6d928fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:05.432516 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-bee8013c-856d-4e6f-828c-a323435d1f16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f31fbf48d684f7b9014d6db0961a9d8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:05.433494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-bee8013c-856d-4e6f-828c-a323435d1f16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f31fbf48d684f7b9014d6db0961a9d8'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:05.455703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c9d3b-aecb-4be6-bf56-7c8ed6d928fe admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:05.462926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d3ddc-ba8a-43d6-b527-3848f2ae61b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:05.464019 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d3ddc-ba8a-43d6-b527-3848f2ae61b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:05.464411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d3ddc-ba8a-43d6-b527-3848f2ae61b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:05.464784 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d3ddc-ba8a-43d6-b527-3848f2ae61b9 admin admin] PATH_INFO: `/v3/projects/cd8910414efa45e2a9341b437f81b2ed/users/9bf9acf3338c45f8b49ea6edef6eddb0/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:05.472656 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1059/2162] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:05 2022] DELETE /identity/v3/users/1f31fbf48d684f7b9014d6db0961a9d8 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:05.473848 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8013c-856d-4e6f-828c-a323435d1f16 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:05.475420 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-bee8013c-856d-4e6f-828c-a323435d1f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:05.490339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d3ddc-ba8a-43d6-b527-3848f2ae61b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8910414efa45e2a9341b437f81b2ed, user_id=9bf9acf3338c45f8b49ea6edef6eddb0, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:05.494399 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d3ddc-ba8a-43d6-b527-3848f2ae61b9 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:05.512426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1104/2163] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:05 2022] PUT /identity/v3/projects/cd8910414efa45e2a9341b437f81b2ed/users/9bf9acf3338c45f8b49ea6edef6eddb0/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:05.513964 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ac70d-3673-486b-a995-8091885d1345 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:05.514253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ac70d-3673-486b-a995-8091885d1345 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:05.514536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ac70d-3673-486b-a995-8091885d1345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:05.536881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d81ac70d-3673-486b-a995-8091885d1345 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:05.538117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d81ac70d-3673-486b-a995-8091885d1345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:05.597164 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:05.635020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8013c-856d-4e6f-828c-a323435d1f16 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:05.670895 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1105/2164] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:05.672240 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ac70d-3673-486b-a995-8091885d1345 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:05.686836 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ac70d-3673-486b-a995-8091885d1345 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:05.697279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:05.701539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a164c4-2584-4559-aff6-774090f4917c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:05.702239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a164c4-2584-4559-aff6-774090f4917c admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:05.702525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a164c4-2584-4559-aff6-774090f4917c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:05.702800 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a164c4-2584-4559-aff6-774090f4917c admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:05.703599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a164c4-2584-4559-aff6-774090f4917c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:05.707836 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a164c4-2584-4559-aff6-774090f4917c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:05.732264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1106/2165] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:05.734411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a164c4-2584-4559-aff6-774090f4917c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:05.742470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-82a164c4-2584-4559-aff6-774090f4917c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:05.789037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734874d-4c4b-4484-826a-709d4ab629da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:05.789037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734874d-4c4b-4484-826a-709d4ab629da admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:05.789037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734874d-4c4b-4484-826a-709d4ab629da admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:05.789037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734874d-4c4b-4484-826a-709d4ab629da admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:05.789479 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734874d-4c4b-4484-826a-709d4ab629da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:05.789479 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734874d-4c4b-4484-826a-709d4ab629da admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:05.819566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1060/2166] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:05.820938 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734874d-4c4b-4484-826a-709d4ab629da admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:05.834251 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734874d-4c4b-4484-826a-709d4ab629da admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:05.856653 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d0841-9794-4f51-be5f-f9825c5313a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:05.859105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d0841-9794-4f51-be5f-f9825c5313a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:05.860420 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d0841-9794-4f51-be5f-f9825c5313a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:05.861603 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d0841-9794-4f51-be5f-f9825c5313a5 admin admin] PATH_INFO: `/v3/projects/cd8910414efa45e2a9341b437f81b2ed/users/9bf9acf3338c45f8b49ea6edef6eddb0/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:05.867263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d0841-9794-4f51-be5f-f9825c5313a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8910414efa45e2a9341b437f81b2ed, user_id=9bf9acf3338c45f8b49ea6edef6eddb0, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:05.876408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d0841-9794-4f51-be5f-f9825c5313a5 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:05.878965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a164c4-2584-4559-aff6-774090f4917c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:05.888845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be7790-a7d6-4848-b52b-fb4ada139485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:05.889529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be7790-a7d6-4848-b52b-fb4ada139485 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:05.889830 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be7790-a7d6-4848-b52b-fb4ada139485 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:05.890109 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be7790-a7d6-4848-b52b-fb4ada139485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:05.905699 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1061/2167] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:05 2022] PUT /identity/v3/projects/cd8910414efa45e2a9341b437f81b2ed/users/9bf9acf3338c45f8b49ea6edef6eddb0/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:05.908305 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d0841-9794-4f51-be5f-f9825c5313a5 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:05.916558 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be7790-a7d6-4848-b52b-fb4ada139485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:05.918457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be7790-a7d6-4848-b52b-fb4ada139485 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:05.949197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1107/2168] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:05.950620 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dab92a-c474-4a33-8806-87048d1c396c None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:05.950911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dab92a-c474-4a33-8806-87048d1c396c None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:05.951192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dab92a-c474-4a33-8806-87048d1c396c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:06.056127 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-41dab92a-c474-4a33-8806-87048d1c396c None None] MFA Rules not processed for user `9bf9acf3338c45f8b49ea6edef6eddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:06.061779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-41dab92a-c474-4a33-8806-87048d1c396c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:06.070980 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d0841-9794-4f51-be5f-f9825c5313a5 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:06.077371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590049c1-fcbf-484d-adc2-e0cd7f88112a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:06.078467 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590049c1-fcbf-484d-adc2-e0cd7f88112a admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:06.078907 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590049c1-fcbf-484d-adc2-e0cd7f88112a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:06.079284 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590049c1-fcbf-484d-adc2-e0cd7f88112a admin admin] PATH_INFO: `/v3/projects/2b46bf2b1b5e4bfd9f3817b8ad5c0110/users/ae1d737f538e4e85b861acb9fbd35542/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:06.097932 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590049c1-fcbf-484d-adc2-e0cd7f88112a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b46bf2b1b5e4bfd9f3817b8ad5c0110, user_id=ae1d737f538e4e85b861acb9fbd35542, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:06.100820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590049c1-fcbf-484d-adc2-e0cd7f88112a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:06.121837 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1062/2169] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:05 2022] PUT /identity/v3/projects/2b46bf2b1b5e4bfd9f3817b8ad5c0110/users/ae1d737f538e4e85b861acb9fbd35542/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:06.138412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e4b81-e18e-47f8-af52-0436defd1525 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:06.139118 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e4b81-e18e-47f8-af52-0436defd1525 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:06.140087 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e4b81-e18e-47f8-af52-0436defd1525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:06.187418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-2a7e4b81-e18e-47f8-af52-0436defd1525 None None] MFA Rules not processed for user `ae1d737f538e4e85b861acb9fbd35542`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:06.189474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-2a7e4b81-e18e-47f8-af52-0436defd1525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:06.264748 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1108/2170] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:06.264748 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dab92a-c474-4a33-8806-87048d1c396c None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:06.266954 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-41dab92a-c474-4a33-8806-87048d1c396c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:06.333762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1063/2171] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:54:06.338680 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e4b81-e18e-47f8-af52-0436defd1525 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:06.360173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dab92a-c474-4a33-8806-87048d1c396c None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:06.377720 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:06.378881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:06.439677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db0890-3f85-430f-8155-d0e71fe7d89f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:06.441216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db0890-3f85-430f-8155-d0e71fe7d89f service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:06.441976 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db0890-3f85-430f-8155-d0e71fe7d89f service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:06.442930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db0890-3f85-430f-8155-d0e71fe7d89f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:06.458334 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e4b81-e18e-47f8-af52-0436defd1525 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:06.471243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196ea12-fd1d-4d59-910b-f9f90a318270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:06.471990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196ea12-fd1d-4d59-910b-f9f90a318270 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:06.472338 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196ea12-fd1d-4d59-910b-f9f90a318270 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:06.472681 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196ea12-fd1d-4d59-910b-f9f90a318270 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:06.473583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196ea12-fd1d-4d59-910b-f9f90a318270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:06.476487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196ea12-fd1d-4d59-910b-f9f90a318270 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:06.521903 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1064/2172] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:06 2022] POST /identity/v3/projects => generated 383 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:06.521903 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196ea12-fd1d-4d59-910b-f9f90a318270 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:06.524807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7196ea12-fd1d-4d59-910b-f9f90a318270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:06.544442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db0890-3f85-430f-8155-d0e71fe7d89f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:06.546703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db0890-3f85-430f-8155-d0e71fe7d89f service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:06.567934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1109/2173] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:06.568983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db0890-3f85-430f-8155-d0e71fe7d89f service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:06.599610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db0890-3f85-430f-8155-d0e71fe7d89f service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:06.640940 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196ea12-fd1d-4d59-910b-f9f90a318270 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:06.649019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895528af-cb42-4831-b01e-39f94a0debc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:06.649973 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895528af-cb42-4831-b01e-39f94a0debc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:06.651005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895528af-cb42-4831-b01e-39f94a0debc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:06.651393 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895528af-cb42-4831-b01e-39f94a0debc7 admin admin] PATH_INFO: `/v3/projects/28bdd0aa542c4816bbcd114b520e1129` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:06.653660 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895528af-cb42-4831-b01e-39f94a0debc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28bdd0aa542c4816bbcd114b520e1129)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:06.656070 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895528af-cb42-4831-b01e-39f94a0debc7 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:06.685518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3af044-86f0-41cd-a1cc-3b572c71e3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:06.686207 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3af044-86f0-41cd-a1cc-3b572c71e3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:06.686492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3af044-86f0-41cd-a1cc-3b572c71e3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:06.686966 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3af044-86f0-41cd-a1cc-3b572c71e3e6 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:06.688356 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3af044-86f0-41cd-a1cc-3b572c71e3e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:06.689699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3af044-86f0-41cd-a1cc-3b572c71e3e6 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:06.720467 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1110/2174] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:06 2022] POST /identity/v3/users => generated 298 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:06.721426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3af044-86f0-41cd-a1cc-3b572c71e3e6 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:06.723140 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-da3af044-86f0-41cd-a1cc-3b572c71e3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:06.731675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-895528af-cb42-4831-b01e-39f94a0debc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28bdd0aa542c4816bbcd114b520e1129', 'request_id': 'req-895528af-cb42-4831-b01e-39f94a0debc7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:06.748947 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-895528af-cb42-4831-b01e-39f94a0debc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28bdd0aa542c4816bbcd114b520e1129', 'request_id': 'req-895528af-cb42-4831-b01e-39f94a0debc7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:06.761528 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-895528af-cb42-4831-b01e-39f94a0debc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28bdd0aa542c4816bbcd114b520e1129', 'request_id': 'req-895528af-cb42-4831-b01e-39f94a0debc7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:06.784427 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-895528af-cb42-4831-b01e-39f94a0debc7 admin admin] The token cache is being invalidate because project 28bdd0aa542c4816bbcd114b520e1129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:06.784726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-895528af-cb42-4831-b01e-39f94a0debc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:06.785525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-895528af-cb42-4831-b01e-39f94a0debc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:06.790706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1065/2175] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:06 2022] DELETE /identity/v3/projects/28bdd0aa542c4816bbcd114b520e1129 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:06.790706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895528af-cb42-4831-b01e-39f94a0debc7 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:06.790706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-895528af-cb42-4831-b01e-39f94a0debc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:06.813695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3af044-86f0-41cd-a1cc-3b572c71e3e6 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:06.820328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:06.850880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:06.876281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42122833-e5da-426b-833f-928e2a104d75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:06.877016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42122833-e5da-426b-833f-928e2a104d75 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:06.877323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42122833-e5da-426b-833f-928e2a104d75 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:06.877627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42122833-e5da-426b-833f-928e2a104d75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:06.877876 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895528af-cb42-4831-b01e-39f94a0debc7 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:06.879467 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-42122833-e5da-426b-833f-928e2a104d75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:06.883623 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed9c05-550a-4e01-86c3-8516aacac8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:06.884420 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed9c05-550a-4e01-86c3-8516aacac8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:06.884710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed9c05-550a-4e01-86c3-8516aacac8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:06.884987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed9c05-550a-4e01-86c3-8516aacac8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:06.885731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed9c05-550a-4e01-86c3-8516aacac8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:06.887370 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed9c05-550a-4e01-86c3-8516aacac8a8 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:06.898402 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1066/2176] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:06.899397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed9c05-550a-4e01-86c3-8516aacac8a8 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:06.901314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-58ed9c05-550a-4e01-86c3-8516aacac8a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:06.914511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed9c05-550a-4e01-86c3-8516aacac8a8 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:06.920816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0308c3-a908-49b2-8154-6ccb9ee07b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:06.922424 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0308c3-a908-49b2-8154-6ccb9ee07b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:06.922974 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0308c3-a908-49b2-8154-6ccb9ee07b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:06.923532 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0308c3-a908-49b2-8154-6ccb9ee07b78 admin admin] PATH_INFO: `/v3/users/7f8b7c8ae31347119a712d4e82a0e30c` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:06.925694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0308c3-a908-49b2-8154-6ccb9ee07b78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f8b7c8ae31347119a712d4e82a0e30c)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:06.930173 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0308c3-a908-49b2-8154-6ccb9ee07b78 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:06.946708 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42122833-e5da-426b-833f-928e2a104d75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:06.948308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42122833-e5da-426b-833f-928e2a104d75 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:06.958954 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1111/2177] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:06.959748 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42122833-e5da-426b-833f-928e2a104d75 service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:06.979961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42122833-e5da-426b-833f-928e2a104d75 service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:06.984958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a957e7-ef9b-436f-a7eb-e7e07ea2d987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:06.985865 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a957e7-ef9b-436f-a7eb-e7e07ea2d987 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:06.986420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a957e7-ef9b-436f-a7eb-e7e07ea2d987 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:06.986698 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a957e7-ef9b-436f-a7eb-e7e07ea2d987 admin admin] PATH_INFO: `/v3/projects/690d4c05819c4a7591cc9c4f0fd12cd3/users/0d52c7c081c2469b9ffeecbb408e4fae/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:07.005337 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9b0308c3-a908-49b2-8154-6ccb9ee07b78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f8b7c8ae31347119a712d4e82a0e30c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:07.026578 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9b0308c3-a908-49b2-8154-6ccb9ee07b78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f8b7c8ae31347119a712d4e82a0e30c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:07.027058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a957e7-ef9b-436f-a7eb-e7e07ea2d987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=690d4c05819c4a7591cc9c4f0fd12cd3, user_id=0d52c7c081c2469b9ffeecbb408e4fae, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:07.027600 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9b0308c3-a908-49b2-8154-6ccb9ee07b78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f8b7c8ae31347119a712d4e82a0e30c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:07.031881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a957e7-ef9b-436f-a7eb-e7e07ea2d987 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:07.037254 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9b0308c3-a908-49b2-8154-6ccb9ee07b78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f8b7c8ae31347119a712d4e82a0e30c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:07.038143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9b0308c3-a908-49b2-8154-6ccb9ee07b78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f8b7c8ae31347119a712d4e82a0e30c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:07.066193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1112/2178] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:06 2022] PUT /identity/v3/projects/690d4c05819c4a7591cc9c4f0fd12cd3/users/0d52c7c081c2469b9ffeecbb408e4fae/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:07.071579 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a957e7-ef9b-436f-a7eb-e7e07ea2d987 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:07.074172 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f0a957e7-ef9b-436f-a7eb-e7e07ea2d987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:07.101663 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1067/2179] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:06 2022] DELETE /identity/v3/users/7f8b7c8ae31347119a712d4e82a0e30c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:07.213676 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a957e7-ef9b-436f-a7eb-e7e07ea2d987 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:07.287284 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:07.316391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b70638-08d0-488a-88c9-87b30eb10ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:07.317841 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b70638-08d0-488a-88c9-87b30eb10ecd admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:07.318140 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b70638-08d0-488a-88c9-87b30eb10ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:07.318419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b70638-08d0-488a-88c9-87b30eb10ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:07.319216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b70638-08d0-488a-88c9-87b30eb10ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:07.321407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b70638-08d0-488a-88c9-87b30eb10ecd admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:07.342467 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1113/2180] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:07.343731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0308c3-a908-49b2-8154-6ccb9ee07b78 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:07.373126 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0308c3-a908-49b2-8154-6ccb9ee07b78 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:07.390205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf94701-545f-458d-aa89-cc6f07c4187b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:07.391656 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf94701-545f-458d-aa89-cc6f07c4187b admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:07.392151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf94701-545f-458d-aa89-cc6f07c4187b admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:07.393445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf94701-545f-458d-aa89-cc6f07c4187b admin admin] PATH_INFO: `/v3/projects/690d4c05819c4a7591cc9c4f0fd12cd3/users/0d52c7c081c2469b9ffeecbb408e4fae/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:07.409369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf94701-545f-458d-aa89-cc6f07c4187b admin admin] RBAC: Authorizing `identity:create_grant(project_id=690d4c05819c4a7591cc9c4f0fd12cd3, user_id=0d52c7c081c2469b9ffeecbb408e4fae, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:07.412243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf94701-545f-458d-aa89-cc6f07c4187b admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:07.450349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1068/2181] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:07 2022] PUT /identity/v3/projects/690d4c05819c4a7591cc9c4f0fd12cd3/users/0d52c7c081c2469b9ffeecbb408e4fae/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:07.527472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57025b-794a-47ce-b935-8d8baa229c35 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:07.527909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57025b-794a-47ce-b935-8d8baa229c35 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:07.528297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57025b-794a-47ce-b935-8d8baa229c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:07.571277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf94701-545f-458d-aa89-cc6f07c4187b admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:07.573717 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ccf94701-545f-458d-aa89-cc6f07c4187b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:07.574917 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-6b57025b-794a-47ce-b935-8d8baa229c35 None None] MFA Rules not processed for user `0d52c7c081c2469b9ffeecbb408e4fae`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:07.576400 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6b57025b-794a-47ce-b935-8d8baa229c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:07.705446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf94701-545f-458d-aa89-cc6f07c4187b admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:07.710496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492899a-2b58-4215-963d-52289d89910d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:07.711447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492899a-2b58-4215-963d-52289d89910d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:07.711851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492899a-2b58-4215-963d-52289d89910d admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:07.712215 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492899a-2b58-4215-963d-52289d89910d admin admin] PATH_INFO: `/v3/projects/93e661cb4a6d4d04a091c528998d06ee` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:07.713550 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492899a-2b58-4215-963d-52289d89910d admin admin] RBAC: Authorizing `identity:delete_project(project_id=93e661cb4a6d4d04a091c528998d06ee)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:07.716316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492899a-2b58-4215-963d-52289d89910d admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:07.717273 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1114/2182] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:07.730771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57025b-794a-47ce-b935-8d8baa229c35 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:07.732319 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6b57025b-794a-47ce-b935-8d8baa229c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:07.786686 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:07.800347 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-5492899a-2b58-4215-963d-52289d89910d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93e661cb4a6d4d04a091c528998d06ee', 'request_id': 'req-5492899a-2b58-4215-963d-52289d89910d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:07.811217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-5492899a-2b58-4215-963d-52289d89910d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93e661cb4a6d4d04a091c528998d06ee', 'request_id': 'req-5492899a-2b58-4215-963d-52289d89910d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:07.824265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57025b-794a-47ce-b935-8d8baa229c35 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:07.825567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-5492899a-2b58-4215-963d-52289d89910d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93e661cb4a6d4d04a091c528998d06ee', 'request_id': 'req-5492899a-2b58-4215-963d-52289d89910d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:07.851754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-5492899a-2b58-4215-963d-52289d89910d admin admin] The token cache is being invalidate because project 93e661cb4a6d4d04a091c528998d06ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:07.851754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-5492899a-2b58-4215-963d-52289d89910d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:07.851754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-5492899a-2b58-4215-963d-52289d89910d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:07.851754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1069/2183] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:07 2022] DELETE /identity/v3/projects/93e661cb4a6d4d04a091c528998d06ee => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:07.861354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14a22b-4c1f-41b4-90da-cd3d7e7ad7bb None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:07.861922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14a22b-4c1f-41b4-90da-cd3d7e7ad7bb None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:07.862209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14a22b-4c1f-41b4-90da-cd3d7e7ad7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:07.883798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-ab14a22b-4c1f-41b4-90da-cd3d7e7ad7bb None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:07.885242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ab14a22b-4c1f-41b4-90da-cd3d7e7ad7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:07.893573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a39f02-7e19-4030-b9bf-920dad4b1a2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:07.894560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a39f02-7e19-4030-b9bf-920dad4b1a2d service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:07.895279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a39f02-7e19-4030-b9bf-920dad4b1a2d service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:07.895736 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a39f02-7e19-4030-b9bf-920dad4b1a2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:07.898506 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-59a39f02-7e19-4030-b9bf-920dad4b1a2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:07.957661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:07.970781 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1070/2184] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:54:07.972352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2a80d-14a8-4430-8d61-64a61a599ab0 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:07.972640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2a80d-14a8-4430-8d61-64a61a599ab0 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:07.972914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2a80d-14a8-4430-8d61-64a61a599ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:07.993335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-5fa2a80d-14a8-4430-8d61-64a61a599ab0 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:07.994582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5fa2a80d-14a8-4430-8d61-64a61a599ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:07.998385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a39f02-7e19-4030-b9bf-920dad4b1a2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:08.004854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a39f02-7e19-4030-b9bf-920dad4b1a2d service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:08.011455 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1071/2185] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:08.014301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1115/2186] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:54:08.015701 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2a80d-14a8-4430-8d61-64a61a599ab0 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:08.028745 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2a80d-14a8-4430-8d61-64a61a599ab0 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:08.034848 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf4c9a-28d0-4105-8fb9-1967ce74ce80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:08.035820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf4c9a-28d0-4105-8fb9-1967ce74ce80 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:08.036349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf4c9a-28d0-4105-8fb9-1967ce74ce80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:08.036842 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf4c9a-28d0-4105-8fb9-1967ce74ce80 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:08.037815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf4c9a-28d0-4105-8fb9-1967ce74ce80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:54:08.038708 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf4c9a-28d0-4105-8fb9-1967ce74ce80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:08.040702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf4c9a-28d0-4105-8fb9-1967ce74ce80 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:08.068944 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1072/2187] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:54:08.071012 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a39f02-7e19-4030-b9bf-920dad4b1a2d service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:08.085101 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a39f02-7e19-4030-b9bf-920dad4b1a2d service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:08.091940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00182a47-4c56-493c-9a3a-8770c7baeb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:08.092695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00182a47-4c56-493c-9a3a-8770c7baeb36 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:08.093023 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00182a47-4c56-493c-9a3a-8770c7baeb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:08.093453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00182a47-4c56-493c-9a3a-8770c7baeb36 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:08.094220 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00182a47-4c56-493c-9a3a-8770c7baeb36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:08.096501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00182a47-4c56-493c-9a3a-8770c7baeb36 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:08.125602 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1116/2188] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:08 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:08.130445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf4c9a-28d0-4105-8fb9-1967ce74ce80 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:08.143225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf4c9a-28d0-4105-8fb9-1967ce74ce80 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:08.213717 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6f643-79fa-4925-ac8d-477dfe81f968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:08.214373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6f643-79fa-4925-ac8d-477dfe81f968 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:08.214605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6f643-79fa-4925-ac8d-477dfe81f968 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:08.215007 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6f643-79fa-4925-ac8d-477dfe81f968 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:08.216041 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6f643-79fa-4925-ac8d-477dfe81f968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:08.218233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6f643-79fa-4925-ac8d-477dfe81f968 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:08.248625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1073/2189] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:08 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:08.249246 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00182a47-4c56-493c-9a3a-8770c7baeb36 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:08.262933 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00182a47-4c56-493c-9a3a-8770c7baeb36 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:08.271187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96022ace-ca90-40ed-85f3-8fcc20aca444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:08.272179 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96022ace-ca90-40ed-85f3-8fcc20aca444 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:08.274062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96022ace-ca90-40ed-85f3-8fcc20aca444 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:08.274619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96022ace-ca90-40ed-85f3-8fcc20aca444 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:08.276623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96022ace-ca90-40ed-85f3-8fcc20aca444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:08.279791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96022ace-ca90-40ed-85f3-8fcc20aca444 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:08.295156 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1117/2190] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:08.300057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6f643-79fa-4925-ac8d-477dfe81f968 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:08.312304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6f643-79fa-4925-ac8d-477dfe81f968 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:08.317296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b520fa-0ca8-4e1a-8743-0897a6f4ded7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:08.317984 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b520fa-0ca8-4e1a-8743-0897a6f4ded7 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:08.318273 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b520fa-0ca8-4e1a-8743-0897a6f4ded7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:08.318556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b520fa-0ca8-4e1a-8743-0897a6f4ded7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:08.319298 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b520fa-0ca8-4e1a-8743-0897a6f4ded7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:08.321490 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b520fa-0ca8-4e1a-8743-0897a6f4ded7 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:08.337779 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1074/2191] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:08.342318 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96022ace-ca90-40ed-85f3-8fcc20aca444 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:08.354558 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96022ace-ca90-40ed-85f3-8fcc20aca444 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:08.368760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ba391-93a1-4ca8-8415-46441c4dad42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:08.368760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ba391-93a1-4ca8-8415-46441c4dad42 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:08.368760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ba391-93a1-4ca8-8415-46441c4dad42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:08.368760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ba391-93a1-4ca8-8415-46441c4dad42 admin admin] PATH_INFO: `/v3/projects/cb3dde49215f47f89b1b1efb5da9abcd/users/cd401d9c1ed148859050e296943e1f78/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:08.390741 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ba391-93a1-4ca8-8415-46441c4dad42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3dde49215f47f89b1b1efb5da9abcd, user_id=cd401d9c1ed148859050e296943e1f78, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:08.393638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ba391-93a1-4ca8-8415-46441c4dad42 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:08.420463 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1118/2192] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:08 2022] PUT /identity/v3/projects/cb3dde49215f47f89b1b1efb5da9abcd/users/cd401d9c1ed148859050e296943e1f78/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:54:08.429080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6be5d-e450-411d-8f29-b56a64ce34e8 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:08.429418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6be5d-e450-411d-8f29-b56a64ce34e8 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:08.429770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6be5d-e450-411d-8f29-b56a64ce34e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:08.475300 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-1ef6be5d-e450-411d-8f29-b56a64ce34e8 None None] MFA Rules not processed for user `cd401d9c1ed148859050e296943e1f78`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:08.477398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1ef6be5d-e450-411d-8f29-b56a64ce34e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:08.492583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:08.558825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1075/2193] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:08.569161 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ba391-93a1-4ca8-8415-46441c4dad42 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:08.641078 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ba391-93a1-4ca8-8415-46441c4dad42 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:08.649878 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9a6b7-368e-49aa-8119-10045b891ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:08.650557 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9a6b7-368e-49aa-8119-10045b891ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:08.650840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9a6b7-368e-49aa-8119-10045b891ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:08.651278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9a6b7-368e-49aa-8119-10045b891ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:08.652148 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9a6b7-368e-49aa-8119-10045b891ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:08.653947 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9a6b7-368e-49aa-8119-10045b891ea9 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:08.683807 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1119/2194] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:08 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:08.687975 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6be5d-e450-411d-8f29-b56a64ce34e8 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:08.699554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6be5d-e450-411d-8f29-b56a64ce34e8 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:08.770679 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55614ac2-e3dc-4d0c-9ee8-569edf001383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:08.771361 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55614ac2-e3dc-4d0c-9ee8-569edf001383 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:08.771645 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55614ac2-e3dc-4d0c-9ee8-569edf001383 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:08.771976 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55614ac2-e3dc-4d0c-9ee8-569edf001383 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:08.772726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55614ac2-e3dc-4d0c-9ee8-569edf001383 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:08.775112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55614ac2-e3dc-4d0c-9ee8-569edf001383 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:08.798661 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1076/2195] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:08 2022] POST /identity/v3/users => generated 296 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:08.803006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9a6b7-368e-49aa-8119-10045b891ea9 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:08.816784 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9a6b7-368e-49aa-8119-10045b891ea9 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:08.822245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39348455-3f2e-4897-8295-31b8cc1654ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:08.822966 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39348455-3f2e-4897-8295-31b8cc1654ef admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:08.823403 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39348455-3f2e-4897-8295-31b8cc1654ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:08.823757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39348455-3f2e-4897-8295-31b8cc1654ef admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:08.824928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39348455-3f2e-4897-8295-31b8cc1654ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:08.829296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39348455-3f2e-4897-8295-31b8cc1654ef admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:08.841500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1120/2196] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:54:08.845685 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55614ac2-e3dc-4d0c-9ee8-569edf001383 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:08.858666 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55614ac2-e3dc-4d0c-9ee8-569edf001383 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:08.864347 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a469cf-194c-428a-bc27-e94779e341fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:08.865343 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a469cf-194c-428a-bc27-e94779e341fd admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:08.865789 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a469cf-194c-428a-bc27-e94779e341fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:08.866288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a469cf-194c-428a-bc27-e94779e341fd admin admin] PATH_INFO: `/v3/projects/59d61f61fc9645b8a85d8624bc2a1851/users/7b8b44144b0747f8b50c84cdf9237333/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:08.892186 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a469cf-194c-428a-bc27-e94779e341fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=59d61f61fc9645b8a85d8624bc2a1851, user_id=7b8b44144b0747f8b50c84cdf9237333, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:08.897348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a469cf-194c-428a-bc27-e94779e341fd admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:08.920398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1077/2197] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:08 2022] PUT /identity/v3/projects/59d61f61fc9645b8a85d8624bc2a1851/users/7b8b44144b0747f8b50c84cdf9237333/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:08.925959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39348455-3f2e-4897-8295-31b8cc1654ef admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:08.941408 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39348455-3f2e-4897-8295-31b8cc1654ef admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:09.007099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:09.023286 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1a451-4584-444b-8234-c924fdcd38a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:09.024482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1a451-4584-444b-8234-c924fdcd38a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:09.024920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1a451-4584-444b-8234-c924fdcd38a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:09.025225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1a451-4584-444b-8234-c924fdcd38a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:09.025974 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1a451-4584-444b-8234-c924fdcd38a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:09.027699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1a451-4584-444b-8234-c924fdcd38a1 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:09.048871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1121/2198] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:54:09.052797 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a469cf-194c-428a-bc27-e94779e341fd admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:09.067774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a469cf-194c-428a-bc27-e94779e341fd admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:09.073551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d8a8de-05f2-46a2-9dfa-6dde21147d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:09.074454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8a8de-05f2-46a2-9dfa-6dde21147d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:09.074734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8a8de-05f2-46a2-9dfa-6dde21147d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:09.075035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8a8de-05f2-46a2-9dfa-6dde21147d03 admin admin] PATH_INFO: `/v3/projects/59d61f61fc9645b8a85d8624bc2a1851/users/7b8b44144b0747f8b50c84cdf9237333/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:09.077127 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d8a8de-05f2-46a2-9dfa-6dde21147d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59d61f61fc9645b8a85d8624bc2a1851, user_id=7b8b44144b0747f8b50c84cdf9237333, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:09.079795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d8a8de-05f2-46a2-9dfa-6dde21147d03 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:09.093127 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1078/2199] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:09 2022] PUT /identity/v3/projects/59d61f61fc9645b8a85d8624bc2a1851/users/7b8b44144b0747f8b50c84cdf9237333/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:09.111746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078a5f0-e592-4541-9054-3ae90e5e1603 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:09.112560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078a5f0-e592-4541-9054-3ae90e5e1603 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:09.113034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078a5f0-e592-4541-9054-3ae90e5e1603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:09.160485 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-f078a5f0-e592-4541-9054-3ae90e5e1603 None None] MFA Rules not processed for user `7b8b44144b0747f8b50c84cdf9237333`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:09.163136 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f078a5f0-e592-4541-9054-3ae90e5e1603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:09.250964 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1122/2200] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:09 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:09.262291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d8a8de-05f2-46a2-9dfa-6dde21147d03 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:09.264084 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-57d8a8de-05f2-46a2-9dfa-6dde21147d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:09.331387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d8a8de-05f2-46a2-9dfa-6dde21147d03 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:09.335250 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37825e3a-9726-4026-a354-5625e54dbbb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:09.336046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37825e3a-9726-4026-a354-5625e54dbbb7 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:09.336430 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37825e3a-9726-4026-a354-5625e54dbbb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:09.336777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37825e3a-9726-4026-a354-5625e54dbbb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:09.347588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37825e3a-9726-4026-a354-5625e54dbbb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:09.349065 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37825e3a-9726-4026-a354-5625e54dbbb7 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:09.360471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1079/2201] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:10.232627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078a5f0-e592-4541-9054-3ae90e5e1603 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:10.310054 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078a5f0-e592-4541-9054-3ae90e5e1603 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:10.315081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae335f-79ce-4938-9b21-87f8aea78425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:10.316044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae335f-79ce-4938-9b21-87f8aea78425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:10.316433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae335f-79ce-4938-9b21-87f8aea78425 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:10.316839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae335f-79ce-4938-9b21-87f8aea78425 admin admin] PATH_INFO: `/v3/users/cd401d9c1ed148859050e296943e1f78` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:10.318762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae335f-79ce-4938-9b21-87f8aea78425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd401d9c1ed148859050e296943e1f78)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:10.321866 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae335f-79ce-4938-9b21-87f8aea78425 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:10.384422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-beae335f-79ce-4938-9b21-87f8aea78425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd401d9c1ed148859050e296943e1f78'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:10.385095 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-beae335f-79ce-4938-9b21-87f8aea78425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd401d9c1ed148859050e296943e1f78'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:10.406524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-beae335f-79ce-4938-9b21-87f8aea78425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd401d9c1ed148859050e296943e1f78'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:10.417121 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-beae335f-79ce-4938-9b21-87f8aea78425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd401d9c1ed148859050e296943e1f78'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:10.425167 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:10.432248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-beae335f-79ce-4938-9b21-87f8aea78425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd401d9c1ed148859050e296943e1f78'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:10.443728 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1123/2202] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:10 2022] DELETE /identity/v3/users/cd401d9c1ed148859050e296943e1f78 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:10.445286 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadffa7-4788-4b3d-bb18-c812c60935f3 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:10.446107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadffa7-4788-4b3d-bb18-c812c60935f3 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:10.446618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadffa7-4788-4b3d-bb18-c812c60935f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:10.481017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-8aadffa7-4788-4b3d-bb18-c812c60935f3 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:10.483027 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8aadffa7-4788-4b3d-bb18-c812c60935f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:10.490403 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:10.578103 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1080/2203] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:10.592352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae335f-79ce-4938-9b21-87f8aea78425 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:10.593999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-beae335f-79ce-4938-9b21-87f8aea78425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:10.670083 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae335f-79ce-4938-9b21-87f8aea78425 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:10.676841 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7593df-2561-4b89-818f-7e9ddbe83eea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:10.678088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7593df-2561-4b89-818f-7e9ddbe83eea service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:10.678210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7593df-2561-4b89-818f-7e9ddbe83eea service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:10.678544 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7593df-2561-4b89-818f-7e9ddbe83eea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:10.694544 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7593df-2561-4b89-818f-7e9ddbe83eea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:10.696762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7593df-2561-4b89-818f-7e9ddbe83eea service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:10.723821 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1124/2204] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:10.845147 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadffa7-4788-4b3d-bb18-c812c60935f3 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:10.847243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8aadffa7-4788-4b3d-bb18-c812c60935f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:10.943554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadffa7-4788-4b3d-bb18-c812c60935f3 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:10.947954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc26928-8d7d-4b41-8c8b-23a46e6fda4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:10.948608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc26928-8d7d-4b41-8c8b-23a46e6fda4c service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:10.948885 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc26928-8d7d-4b41-8c8b-23a46e6fda4c service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:10.949156 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc26928-8d7d-4b41-8c8b-23a46e6fda4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:10.950953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5bc26928-8d7d-4b41-8c8b-23a46e6fda4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:11.022162 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc26928-8d7d-4b41-8c8b-23a46e6fda4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:11.024674 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc26928-8d7d-4b41-8c8b-23a46e6fda4c service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:11.035230 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1081/2205] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:11.220677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7593df-2561-4b89-818f-7e9ddbe83eea service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:11.224351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fe7593df-2561-4b89-818f-7e9ddbe83eea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:11.271944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7593df-2561-4b89-818f-7e9ddbe83eea service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:11.282193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:11.284038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:11.284918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:11.285342 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4 admin admin] PATH_INFO: `/v3/projects/cb3dde49215f47f89b1b1efb5da9abcd` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:11.286831 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb3dde49215f47f89b1b1efb5da9abcd)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:11.288932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:11.369202 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb3dde49215f47f89b1b1efb5da9abcd', 'request_id': 'req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:11.382661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb3dde49215f47f89b1b1efb5da9abcd', 'request_id': 'req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:11.392833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb3dde49215f47f89b1b1efb5da9abcd', 'request_id': 'req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:11.424134 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4 admin admin] The token cache is being invalidate because project cb3dde49215f47f89b1b1efb5da9abcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:11.424954 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:11.425865 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:11.428554 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1125/2206] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:11 2022] DELETE /identity/v3/projects/cb3dde49215f47f89b1b1efb5da9abcd => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:11.432857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc26928-8d7d-4b41-8c8b-23a46e6fda4c service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:11.444307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc26928-8d7d-4b41-8c8b-23a46e6fda4c service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:11.561161 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fae85-4e61-477a-a6f2-cf5e0c3f738d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:11.562616 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fae85-4e61-477a-a6f2-cf5e0c3f738d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:11.562912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fae85-4e61-477a-a6f2-cf5e0c3f738d admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:11.563530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fae85-4e61-477a-a6f2-cf5e0c3f738d admin admin] PATH_INFO: `/v3/users/7b8b44144b0747f8b50c84cdf9237333` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:11.565425 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40fae85-4e61-477a-a6f2-cf5e0c3f738d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b8b44144b0747f8b50c84cdf9237333)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:11.567864 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40fae85-4e61-477a-a6f2-cf5e0c3f738d admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:11.578230 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:11.691260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d40fae85-4e61-477a-a6f2-cf5e0c3f738d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b8b44144b0747f8b50c84cdf9237333'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:11.700277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d40fae85-4e61-477a-a6f2-cf5e0c3f738d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b8b44144b0747f8b50c84cdf9237333'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:11.702992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d40fae85-4e61-477a-a6f2-cf5e0c3f738d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b8b44144b0747f8b50c84cdf9237333'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:11.714877 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d40fae85-4e61-477a-a6f2-cf5e0c3f738d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b8b44144b0747f8b50c84cdf9237333'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:11.715849 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d40fae85-4e61-477a-a6f2-cf5e0c3f738d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b8b44144b0747f8b50c84cdf9237333'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:11.761394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1082/2207] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:11 2022] DELETE /identity/v3/users/7b8b44144b0747f8b50c84cdf9237333 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:11.879170 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:11.881817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:11.994199 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b32d9-35ce-4f0c-8a6b-e08f103dfcc4 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:12.042922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:12.079828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399696e-41b2-4303-81d3-88abae6dfc69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:12.080741 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399696e-41b2-4303-81d3-88abae6dfc69 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:12.081917 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399696e-41b2-4303-81d3-88abae6dfc69 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:12.082663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399696e-41b2-4303-81d3-88abae6dfc69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:12.086998 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6399696e-41b2-4303-81d3-88abae6dfc69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:12.185997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fae85-4e61-477a-a6f2-cf5e0c3f738d admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:12.187627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d40fae85-4e61-477a-a6f2-cf5e0c3f738d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:12.260903 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399696e-41b2-4303-81d3-88abae6dfc69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:12.263777 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399696e-41b2-4303-81d3-88abae6dfc69 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:12.310539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1126/2208] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:12.313264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399696e-41b2-4303-81d3-88abae6dfc69 service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:12.316528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6399696e-41b2-4303-81d3-88abae6dfc69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:12.350068 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fae85-4e61-477a-a6f2-cf5e0c3f738d admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:12.365859 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc276fc1-d4e9-4418-b8d0-a74da007536f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:12.368290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc276fc1-d4e9-4418-b8d0-a74da007536f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:12.371304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc276fc1-d4e9-4418-b8d0-a74da007536f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:12.377370 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc276fc1-d4e9-4418-b8d0-a74da007536f admin admin] PATH_INFO: `/v3/projects/59d61f61fc9645b8a85d8624bc2a1851` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:12.378761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc276fc1-d4e9-4418-b8d0-a74da007536f admin admin] RBAC: Authorizing `identity:delete_project(project_id=59d61f61fc9645b8a85d8624bc2a1851)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:12.381174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc276fc1-d4e9-4418-b8d0-a74da007536f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:12.465866 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399696e-41b2-4303-81d3-88abae6dfc69 service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:12.489721 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-fc276fc1-d4e9-4418-b8d0-a74da007536f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59d61f61fc9645b8a85d8624bc2a1851', 'request_id': 'req-fc276fc1-d4e9-4418-b8d0-a74da007536f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:12.512257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-fc276fc1-d4e9-4418-b8d0-a74da007536f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59d61f61fc9645b8a85d8624bc2a1851', 'request_id': 'req-fc276fc1-d4e9-4418-b8d0-a74da007536f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:12.523402 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-fc276fc1-d4e9-4418-b8d0-a74da007536f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59d61f61fc9645b8a85d8624bc2a1851', 'request_id': 'req-fc276fc1-d4e9-4418-b8d0-a74da007536f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:12.551092 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-fc276fc1-d4e9-4418-b8d0-a74da007536f admin admin] The token cache is being invalidate because project 59d61f61fc9645b8a85d8624bc2a1851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:12.551915 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-fc276fc1-d4e9-4418-b8d0-a74da007536f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:12.553377 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-fc276fc1-d4e9-4418-b8d0-a74da007536f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:12.557235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1083/2209] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:12 2022] DELETE /identity/v3/projects/59d61f61fc9645b8a85d8624bc2a1851 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:54:12.563597 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500305f-5bf0-4c3d-bc12-2dbf4e9b1776 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:12.564352 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500305f-5bf0-4c3d-bc12-2dbf4e9b1776 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:12.564682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500305f-5bf0-4c3d-bc12-2dbf4e9b1776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:12.594107 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93c3da-d004-40dd-b80d-ccd591612d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:12.595411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93c3da-d004-40dd-b80d-ccd591612d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:12.595957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93c3da-d004-40dd-b80d-ccd591612d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:12.596750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93c3da-d004-40dd-b80d-ccd591612d55 admin admin] PATH_INFO: `/v3/users/b91baa8b29674a46a9a40f2c8f5de355` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:12.598695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93c3da-d004-40dd-b80d-ccd591612d55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b91baa8b29674a46a9a40f2c8f5de355)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:12.602111 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93c3da-d004-40dd-b80d-ccd591612d55 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:12.607882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-9500305f-5bf0-4c3d-bc12-2dbf4e9b1776 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:12.610603 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-9500305f-5bf0-4c3d-bc12-2dbf4e9b1776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:12.623160 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1084/2210] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:12.633590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe184297-20f3-45d0-9a21-9f799129f747 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:12.633852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe184297-20f3-45d0-9a21-9f799129f747 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:12.634121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe184297-20f3-45d0-9a21-9f799129f747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:12.673302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:12.690012 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-fe184297-20f3-45d0-9a21-9f799129f747 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:12.701477 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:12.720042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-fe184297-20f3-45d0-9a21-9f799129f747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:12.722673 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1a93c3da-d004-40dd-b80d-ccd591612d55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b91baa8b29674a46a9a40f2c8f5de355'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:12.723769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1a93c3da-d004-40dd-b80d-ccd591612d55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b91baa8b29674a46a9a40f2c8f5de355'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:12.758998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1085/2211] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:12.758998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe184297-20f3-45d0-9a21-9f799129f747 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:12.766485 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe184297-20f3-45d0-9a21-9f799129f747 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:12.767340 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1a93c3da-d004-40dd-b80d-ccd591612d55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b91baa8b29674a46a9a40f2c8f5de355'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:12.777542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3f7ae-f4aa-4de0-90fa-234718e98e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:12.778622 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3f7ae-f4aa-4de0-90fa-234718e98e30 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:12.779100 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3f7ae-f4aa-4de0-90fa-234718e98e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:12.779527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3f7ae-f4aa-4de0-90fa-234718e98e30 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:12.780510 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3f7ae-f4aa-4de0-90fa-234718e98e30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:54:12.780924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3f7ae-f4aa-4de0-90fa-234718e98e30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:12.783339 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3f7ae-f4aa-4de0-90fa-234718e98e30 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:12.788000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1a93c3da-d004-40dd-b80d-ccd591612d55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b91baa8b29674a46a9a40f2c8f5de355'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:12.804929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1a93c3da-d004-40dd-b80d-ccd591612d55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b91baa8b29674a46a9a40f2c8f5de355'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:12.813892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1086/2212] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:54:12.818388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1127/2213] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:12 2022] DELETE /identity/v3/users/b91baa8b29674a46a9a40f2c8f5de355 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:12.818803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3f7ae-f4aa-4de0-90fa-234718e98e30 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:12.826599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e2b7f-e358-47e5-9cf3-9147094e94ff None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:12.826907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e2b7f-e358-47e5-9cf3-9147094e94ff None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:12.827193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e2b7f-e358-47e5-9cf3-9147094e94ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:12.849719 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3f7ae-f4aa-4de0-90fa-234718e98e30 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:12.886559 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-b74e2b7f-e358-47e5-9cf3-9147094e94ff None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:12.889552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b74e2b7f-e358-47e5-9cf3-9147094e94ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:13.020836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46f982-aaaa-4606-a7cc-3b72e19c7c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:13.021518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46f982-aaaa-4606-a7cc-3b72e19c7c3f admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:13.024221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46f982-aaaa-4606-a7cc-3b72e19c7c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:13.025256 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46f982-aaaa-4606-a7cc-3b72e19c7c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:13.026181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46f982-aaaa-4606-a7cc-3b72e19c7c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:13.028642 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46f982-aaaa-4606-a7cc-3b72e19c7c3f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:13.069682 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1128/2215] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:54:13.069896 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1087/2215] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:12 2022] POST /identity/v3/projects => generated 375 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:13.076962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e2b7f-e358-47e5-9cf3-9147094e94ff None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:13.081050 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46f982-aaaa-4606-a7cc-3b72e19c7c3f admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:13.082973 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-aa46f982-aaaa-4606-a7cc-3b72e19c7c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:13.091825 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e2b7f-e358-47e5-9cf3-9147094e94ff None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:13.162703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91ba80-e043-4d9d-8b98-83b987cf04d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:13.164487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91ba80-e043-4d9d-8b98-83b987cf04d7 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:13.165104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91ba80-e043-4d9d-8b98-83b987cf04d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:13.165508 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91ba80-e043-4d9d-8b98-83b987cf04d7 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:13.166889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91ba80-e043-4d9d-8b98-83b987cf04d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:13.169141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91ba80-e043-4d9d-8b98-83b987cf04d7 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:13.173349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46f982-aaaa-4606-a7cc-3b72e19c7c3f admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:13.187157 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccc47c-af4e-4e18-a45d-dd2cf62f46a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:13.187930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccc47c-af4e-4e18-a45d-dd2cf62f46a5 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:13.188213 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccc47c-af4e-4e18-a45d-dd2cf62f46a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:13.188468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccc47c-af4e-4e18-a45d-dd2cf62f46a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:13.199948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccc47c-af4e-4e18-a45d-dd2cf62f46a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:13.201484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccc47c-af4e-4e18-a45d-dd2cf62f46a5 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:13.206591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1129/2216] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:13 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:13.208657 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91ba80-e043-4d9d-8b98-83b987cf04d7 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:13.214801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1088/2217] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:13.222569 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91ba80-e043-4d9d-8b98-83b987cf04d7 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:13.228876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a2c89-8b3f-4aa5-8502-e00e24f361b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:13.229730 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a2c89-8b3f-4aa5-8502-e00e24f361b4 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:13.230328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a2c89-8b3f-4aa5-8502-e00e24f361b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:13.230681 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a2c89-8b3f-4aa5-8502-e00e24f361b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:13.231835 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a2c89-8b3f-4aa5-8502-e00e24f361b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:13.233680 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a2c89-8b3f-4aa5-8502-e00e24f361b4 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:13.246885 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1130/2218] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:13.250910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccc47c-af4e-4e18-a45d-dd2cf62f46a5 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:13.261258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccc47c-af4e-4e18-a45d-dd2cf62f46a5 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:13.266196 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bd977-fc22-415e-9cfe-c8183111038e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:13.266887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bd977-fc22-415e-9cfe-c8183111038e admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:13.267188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bd977-fc22-415e-9cfe-c8183111038e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:13.267479 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bd977-fc22-415e-9cfe-c8183111038e admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:13.268262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10bd977-fc22-415e-9cfe-c8183111038e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:13.269946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10bd977-fc22-415e-9cfe-c8183111038e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:13.285637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1089/2219] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:13.290881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a2c89-8b3f-4aa5-8502-e00e24f361b4 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:13.302722 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a2c89-8b3f-4aa5-8502-e00e24f361b4 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:13.307743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd2fff-8e32-40b7-a0ca-026bb69b353c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:13.309111 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd2fff-8e32-40b7-a0ca-026bb69b353c admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:13.309388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd2fff-8e32-40b7-a0ca-026bb69b353c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:13.309663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd2fff-8e32-40b7-a0ca-026bb69b353c admin admin] PATH_INFO: `/v3/projects/972647dccecd4053acc4cfb3b4d526f7/users/420ecfdfc7f14d49a816d1506e8c4e7d/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:13.325659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd2fff-8e32-40b7-a0ca-026bb69b353c admin admin] RBAC: Authorizing `identity:create_grant(project_id=972647dccecd4053acc4cfb3b4d526f7, user_id=420ecfdfc7f14d49a816d1506e8c4e7d, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:13.328373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd2fff-8e32-40b7-a0ca-026bb69b353c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:13.344012 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1131/2220] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:13 2022] PUT /identity/v3/projects/972647dccecd4053acc4cfb3b4d526f7/users/420ecfdfc7f14d49a816d1506e8c4e7d/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:13.356926 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c8865-0ae3-4995-8356-2a1c3e2f2815 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:13.357115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c8865-0ae3-4995-8356-2a1c3e2f2815 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:13.357395 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c8865-0ae3-4995-8356-2a1c3e2f2815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:13.386072 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-215c8865-0ae3-4995-8356-2a1c3e2f2815 None None] MFA Rules not processed for user `420ecfdfc7f14d49a816d1506e8c4e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:13.387116 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-215c8865-0ae3-4995-8356-2a1c3e2f2815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:13.446832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:13.458942 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1090/2221] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:13.464066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd2fff-8e32-40b7-a0ca-026bb69b353c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:13.473386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:13.509293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c8865-0ae3-4995-8356-2a1c3e2f2815 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:13.510822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-215c8865-0ae3-4995-8356-2a1c3e2f2815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:13.564339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd2fff-8e32-40b7-a0ca-026bb69b353c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:13.571188 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13893344-78c4-4459-83b8-1f521bb254bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:13.571895 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13893344-78c4-4459-83b8-1f521bb254bb admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:13.572179 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13893344-78c4-4459-83b8-1f521bb254bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:13.572481 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13893344-78c4-4459-83b8-1f521bb254bb admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:13.573188 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13893344-78c4-4459-83b8-1f521bb254bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:13.575088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13893344-78c4-4459-83b8-1f521bb254bb admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:13.592108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c8865-0ae3-4995-8356-2a1c3e2f2815 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:13.597754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:13.598672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:13.599101 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:13.599309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0 admin admin] PATH_INFO: `/v3/projects/04ec27237ddc47b09a744d525e107c85` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:13.600670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04ec27237ddc47b09a744d525e107c85)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:13.602603 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:13.608695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1132/2222] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:13 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:13.613043 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13893344-78c4-4459-83b8-1f521bb254bb admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:13.662246 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04ec27237ddc47b09a744d525e107c85', 'request_id': 'req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:13.673294 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04ec27237ddc47b09a744d525e107c85', 'request_id': 'req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:13.680785 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04ec27237ddc47b09a744d525e107c85', 'request_id': 'req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:13.694494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13893344-78c4-4459-83b8-1f521bb254bb admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:13.695256 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0 admin admin] The token cache is being invalidate because project 04ec27237ddc47b09a744d525e107c85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:13.695483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:13.696554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:13.698231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1091/2223] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:13 2022] DELETE /identity/v3/projects/04ec27237ddc47b09a744d525e107c85 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:13.702062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:13.703606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:13.787789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd779326-0f26-4f5a-a99f-4cb33d238d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:13.788949 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd779326-0f26-4f5a-a99f-4cb33d238d54 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:13.789314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd779326-0f26-4f5a-a99f-4cb33d238d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:13.789600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd779326-0f26-4f5a-a99f-4cb33d238d54 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:13.790341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd779326-0f26-4f5a-a99f-4cb33d238d54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:13.792313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd779326-0f26-4f5a-a99f-4cb33d238d54 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:13.812034 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e5c5f-4055-4de1-a93b-8ed7d88b91b0 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:13.817168 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1575c-b0f1-4809-8fbe-8882d93e4d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:13.818073 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1575c-b0f1-4809-8fbe-8882d93e4d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:13.818358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1575c-b0f1-4809-8fbe-8882d93e4d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:13.818637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1575c-b0f1-4809-8fbe-8882d93e4d69 admin admin] PATH_INFO: `/v3/users/9bf9acf3338c45f8b49ea6edef6eddb0` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:13.820698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1575c-b0f1-4809-8fbe-8882d93e4d69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf9acf3338c45f8b49ea6edef6eddb0)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:13.822919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1575c-b0f1-4809-8fbe-8882d93e4d69 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:13.825111 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1133/2224] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:13 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:54:13.829288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd779326-0f26-4f5a-a99f-4cb33d238d54 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:13.830965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-bd779326-0f26-4f5a-a99f-4cb33d238d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:13.845416 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd779326-0f26-4f5a-a99f-4cb33d238d54 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:13.850975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f1386-d439-4b3b-bb67-66cfaebeeb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:13.851847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f1386-d439-4b3b-bb67-66cfaebeeb8c admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:13.852407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f1386-d439-4b3b-bb67-66cfaebeeb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:13.852525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f1386-d439-4b3b-bb67-66cfaebeeb8c admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:13.853344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f1386-d439-4b3b-bb67-66cfaebeeb8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:13.855253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f1386-d439-4b3b-bb67-66cfaebeeb8c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:13.862995 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:13.871248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1134/2225] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:54:13.872183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f1386-d439-4b3b-bb67-66cfaebeeb8c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:13.884547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f1386-d439-4b3b-bb67-66cfaebeeb8c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:13.890622 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8d3be-c9b2-44a8-9528-1d6b539a22f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:13.891559 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8d3be-c9b2-44a8-9528-1d6b539a22f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:13.891825 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8d3be-c9b2-44a8-9528-1d6b539a22f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:13.892107 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8d3be-c9b2-44a8-9528-1d6b539a22f6 admin admin] PATH_INFO: `/v3/projects/ee0e855346524a838236dc89e2eccfe4/users/21e6bacbf6014abfae46bdea300d4a29/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:13.905806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-e2d1575c-b0f1-4809-8fbe-8882d93e4d69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf9acf3338c45f8b49ea6edef6eddb0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:13.911864 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8d3be-c9b2-44a8-9528-1d6b539a22f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee0e855346524a838236dc89e2eccfe4, user_id=21e6bacbf6014abfae46bdea300d4a29, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:13.915218 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8d3be-c9b2-44a8-9528-1d6b539a22f6 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:13.918027 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:13.918324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-e2d1575c-b0f1-4809-8fbe-8882d93e4d69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf9acf3338c45f8b49ea6edef6eddb0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:13.919531 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-e2d1575c-b0f1-4809-8fbe-8882d93e4d69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf9acf3338c45f8b49ea6edef6eddb0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:13.932774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-e2d1575c-b0f1-4809-8fbe-8882d93e4d69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf9acf3338c45f8b49ea6edef6eddb0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:13.933539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-e2d1575c-b0f1-4809-8fbe-8882d93e4d69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf9acf3338c45f8b49ea6edef6eddb0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:13.937389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1135/2226] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:13 2022] PUT /identity/v3/projects/ee0e855346524a838236dc89e2eccfe4/users/21e6bacbf6014abfae46bdea300d4a29/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:13.941454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8d3be-c9b2-44a8-9528-1d6b539a22f6 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:13.946173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1be8d3be-c9b2-44a8-9528-1d6b539a22f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:13.958378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1092/2227] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:13 2022] DELETE /identity/v3/users/9bf9acf3338c45f8b49ea6edef6eddb0 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:54:14.017956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8d3be-c9b2-44a8-9528-1d6b539a22f6 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:14.105075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05821ec-5be1-46f5-823b-82d1452df9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:14.107151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05821ec-5be1-46f5-823b-82d1452df9f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:14.107616 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05821ec-5be1-46f5-823b-82d1452df9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:14.108108 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05821ec-5be1-46f5-823b-82d1452df9f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:14.109128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05821ec-5be1-46f5-823b-82d1452df9f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:14.111415 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05821ec-5be1-46f5-823b-82d1452df9f5 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:14.136915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1136/2228] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:14.139124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1575c-b0f1-4809-8fbe-8882d93e4d69 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:14.150441 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1575c-b0f1-4809-8fbe-8882d93e4d69 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:14.156319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e5e6a-e464-45d2-8d87-026cc40bec35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:14.157697 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e5e6a-e464-45d2-8d87-026cc40bec35 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:14.157697 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e5e6a-e464-45d2-8d87-026cc40bec35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:14.157923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e5e6a-e464-45d2-8d87-026cc40bec35 admin admin] PATH_INFO: `/v3/projects/ee0e855346524a838236dc89e2eccfe4/users/21e6bacbf6014abfae46bdea300d4a29/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:14.160525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e5e6a-e464-45d2-8d87-026cc40bec35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee0e855346524a838236dc89e2eccfe4, user_id=21e6bacbf6014abfae46bdea300d4a29, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:14.163654 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e5e6a-e464-45d2-8d87-026cc40bec35 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:14.175773 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1093/2229] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:14 2022] PUT /identity/v3/projects/ee0e855346524a838236dc89e2eccfe4/users/21e6bacbf6014abfae46bdea300d4a29/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:14.189370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de91a29-4fb2-442a-b615-c003d99b2a1b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:14.189666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de91a29-4fb2-442a-b615-c003d99b2a1b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:14.190713 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de91a29-4fb2-442a-b615-c003d99b2a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:14.230537 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-1de91a29-4fb2-442a-b615-c003d99b2a1b None None] MFA Rules not processed for user `21e6bacbf6014abfae46bdea300d4a29`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:14.232302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1de91a29-4fb2-442a-b615-c003d99b2a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:14.285098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e5e6a-e464-45d2-8d87-026cc40bec35 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:14.287096 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-414e5e6a-e464-45d2-8d87-026cc40bec35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:14.373269 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1137/2230] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:54:14.385857 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de91a29-4fb2-442a-b615-c003d99b2a1b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:14.387483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1de91a29-4fb2-442a-b615-c003d99b2a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:14.387902 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e5e6a-e464-45d2-8d87-026cc40bec35 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:14.394757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b894df-cfb7-400f-b719-96b268c8b6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:14.395637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b894df-cfb7-400f-b719-96b268c8b6f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:14.395955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b894df-cfb7-400f-b719-96b268c8b6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:14.399876 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b894df-cfb7-400f-b719-96b268c8b6f2 admin admin] PATH_INFO: `/v3/projects/cd8910414efa45e2a9341b437f81b2ed` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:14.401288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b894df-cfb7-400f-b719-96b268c8b6f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8910414efa45e2a9341b437f81b2ed)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:14.406031 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b894df-cfb7-400f-b719-96b268c8b6f2 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:14.426435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:14.469727 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de91a29-4fb2-442a-b615-c003d99b2a1b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:14.476535 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834f592-8c2c-4c69-86bd-8b95602fba43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:14.476817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834f592-8c2c-4c69-86bd-8b95602fba43 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:14.477021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834f592-8c2c-4c69-86bd-8b95602fba43 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:14.477202 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834f592-8c2c-4c69-86bd-8b95602fba43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:14.479100 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8834f592-8c2c-4c69-86bd-8b95602fba43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:14.491337 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-69b894df-cfb7-400f-b719-96b268c8b6f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8910414efa45e2a9341b437f81b2ed', 'request_id': 'req-69b894df-cfb7-400f-b719-96b268c8b6f2'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:14.504719 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-69b894df-cfb7-400f-b719-96b268c8b6f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8910414efa45e2a9341b437f81b2ed', 'request_id': 'req-69b894df-cfb7-400f-b719-96b268c8b6f2'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:14.514239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-69b894df-cfb7-400f-b719-96b268c8b6f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8910414efa45e2a9341b437f81b2ed', 'request_id': 'req-69b894df-cfb7-400f-b719-96b268c8b6f2'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:14.530046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-69b894df-cfb7-400f-b719-96b268c8b6f2 admin admin] The token cache is being invalidate because project cd8910414efa45e2a9341b437f81b2ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:14.530463 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-69b894df-cfb7-400f-b719-96b268c8b6f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:14.531567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-69b894df-cfb7-400f-b719-96b268c8b6f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:14.536123 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1094/2231] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:14 2022] DELETE /identity/v3/projects/cd8910414efa45e2a9341b437f81b2ed => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:54:14.542328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6db06b-f7a1-45d9-bad3-a94f49e30da6 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:14.542683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6db06b-f7a1-45d9-bad3-a94f49e30da6 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:14.543018 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6db06b-f7a1-45d9-bad3-a94f49e30da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:14.566557 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834f592-8c2c-4c69-86bd-8b95602fba43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:14.570232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-7d6db06b-f7a1-45d9-bad3-a94f49e30da6 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:14.570623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834f592-8c2c-4c69-86bd-8b95602fba43 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:14.572109 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7d6db06b-f7a1-45d9-bad3-a94f49e30da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:14.588339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1138/2232] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:54:14.597540 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:14.659762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1095/2233] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:54:14.668871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d847e-7270-48c5-a82b-762c4a43e815 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:14.669399 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d847e-7270-48c5-a82b-762c4a43e815 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:14.669922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d847e-7270-48c5-a82b-762c4a43e815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:14.689437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-8c6d847e-7270-48c5-a82b-762c4a43e815 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:14.691973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8c6d847e-7270-48c5-a82b-762c4a43e815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:14.704875 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1139/2234] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:14.706546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6db06b-f7a1-45d9-bad3-a94f49e30da6 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:14.716798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6db06b-f7a1-45d9-bad3-a94f49e30da6 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:14.721551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3208cd-746c-4eaf-a35c-8541337f3eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:14.722213 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3208cd-746c-4eaf-a35c-8541337f3eba admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:14.722497 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3208cd-746c-4eaf-a35c-8541337f3eba admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:14.722776 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3208cd-746c-4eaf-a35c-8541337f3eba admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:14.723516 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3208cd-746c-4eaf-a35c-8541337f3eba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:54:14.723855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3208cd-746c-4eaf-a35c-8541337f3eba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:14.725457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3208cd-746c-4eaf-a35c-8541337f3eba admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:14.740946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1096/2235] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:54:14.745375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d847e-7270-48c5-a82b-762c4a43e815 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:14.756301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d847e-7270-48c5-a82b-762c4a43e815 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:14.762193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9db8b-96f8-4eac-b397-6249b5f35a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:14.762875 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9db8b-96f8-4eac-b397-6249b5f35a6d admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:14.763153 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9db8b-96f8-4eac-b397-6249b5f35a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:14.763440 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9db8b-96f8-4eac-b397-6249b5f35a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:14.764246 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9db8b-96f8-4eac-b397-6249b5f35a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:14.766374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9db8b-96f8-4eac-b397-6249b5f35a6d admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:14.801163 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1140/2236] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:14 2022] POST /identity/v3/projects => generated 399 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:14.802360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3208cd-746c-4eaf-a35c-8541337f3eba admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:14.822471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3208cd-746c-4eaf-a35c-8541337f3eba admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:14.947586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ec103-0024-4405-a667-4e049d8b620b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:14.947586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ec103-0024-4405-a667-4e049d8b620b admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:14.947586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ec103-0024-4405-a667-4e049d8b620b admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:14.947586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ec103-0024-4405-a667-4e049d8b620b admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:14.948108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ec103-0024-4405-a667-4e049d8b620b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:14.954380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ec103-0024-4405-a667-4e049d8b620b admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:14.986562 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1097/2237] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:14 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:14.990339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9db8b-96f8-4eac-b397-6249b5f35a6d admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:15.001672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9db8b-96f8-4eac-b397-6249b5f35a6d admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:15.006896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42ef0d-cfde-4872-9122-4069fa9b4f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:15.007799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42ef0d-cfde-4872-9122-4069fa9b4f71 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:15.018812 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42ef0d-cfde-4872-9122-4069fa9b4f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:15.019153 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42ef0d-cfde-4872-9122-4069fa9b4f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:15.019934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42ef0d-cfde-4872-9122-4069fa9b4f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:15.021696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42ef0d-cfde-4872-9122-4069fa9b4f71 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:15.037077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1141/2238] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:15.037953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ec103-0024-4405-a667-4e049d8b620b admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:15.049481 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ec103-0024-4405-a667-4e049d8b620b admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:15.055795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce01e9-0df6-4f47-b5bb-7112079e9046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:15.056741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce01e9-0df6-4f47-b5bb-7112079e9046 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:15.057131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce01e9-0df6-4f47-b5bb-7112079e9046 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:15.057338 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce01e9-0df6-4f47-b5bb-7112079e9046 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:15.058232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce01e9-0df6-4f47-b5bb-7112079e9046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:15.060143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce01e9-0df6-4f47-b5bb-7112079e9046 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:15.073334 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1098/2239] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:54:15.082275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42ef0d-cfde-4872-9122-4069fa9b4f71 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:15.095843 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42ef0d-cfde-4872-9122-4069fa9b4f71 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:15.102747 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519fb11-4ba9-4b4e-877c-42ee0502248d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:15.104302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519fb11-4ba9-4b4e-877c-42ee0502248d admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:15.104910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519fb11-4ba9-4b4e-877c-42ee0502248d admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:15.105349 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519fb11-4ba9-4b4e-877c-42ee0502248d admin admin] PATH_INFO: `/v3/projects/de0fa93dd8ed4ff094790c90d27faf51/users/de1399f02bdf42c6980a850424cb4019/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:15.125028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b519fb11-4ba9-4b4e-877c-42ee0502248d admin admin] RBAC: Authorizing `identity:create_grant(project_id=de0fa93dd8ed4ff094790c90d27faf51, user_id=de1399f02bdf42c6980a850424cb4019, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:15.128519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b519fb11-4ba9-4b4e-877c-42ee0502248d admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:15.147909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1142/2240] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:15 2022] PUT /identity/v3/projects/de0fa93dd8ed4ff094790c90d27faf51/users/de1399f02bdf42c6980a850424cb4019/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:15.159062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3ab76-6b75-4edb-9b99-cd2f6b2621dc None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:15.159778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3ab76-6b75-4edb-9b99-cd2f6b2621dc None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:15.160294 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3ab76-6b75-4edb-9b99-cd2f6b2621dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:15.191262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-f5f3ab76-6b75-4edb-9b99-cd2f6b2621dc None None] MFA Rules not processed for user `de1399f02bdf42c6980a850424cb4019`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:15.193037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f5f3ab76-6b75-4edb-9b99-cd2f6b2621dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:15.283318 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1099/2241] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:15.288758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519fb11-4ba9-4b4e-877c-42ee0502248d admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:15.388788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519fb11-4ba9-4b4e-877c-42ee0502248d admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:15.394406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736bbe5-e213-4876-9209-4a3e0d19a6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:15.395026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736bbe5-e213-4876-9209-4a3e0d19a6d0 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:15.395335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736bbe5-e213-4876-9209-4a3e0d19a6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:15.395582 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736bbe5-e213-4876-9209-4a3e0d19a6d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:15.396328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736bbe5-e213-4876-9209-4a3e0d19a6d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:15.398577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736bbe5-e213-4876-9209-4a3e0d19a6d0 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:15.425888 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:15.438090 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1143/2242] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:15 2022] POST /identity/v3/projects => generated 401 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:54:15.439464 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3ab76-6b75-4edb-9b99-cd2f6b2621dc None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:15.450895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3ab76-6b75-4edb-9b99-cd2f6b2621dc None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:15.462905 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:15.511502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b777cb-a1b9-4b32-b4e9-9a0502ada87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:15.512348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b777cb-a1b9-4b32-b4e9-9a0502ada87d admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:15.512655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b777cb-a1b9-4b32-b4e9-9a0502ada87d admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:15.512949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b777cb-a1b9-4b32-b4e9-9a0502ada87d admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:15.513748 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b777cb-a1b9-4b32-b4e9-9a0502ada87d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:15.515921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b777cb-a1b9-4b32-b4e9-9a0502ada87d admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:15.539691 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1100/2243] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:15 2022] POST /identity/v3/users => generated 307 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:54:15.541152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736bbe5-e213-4876-9209-4a3e0d19a6d0 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:15.551989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736bbe5-e213-4876-9209-4a3e0d19a6d0 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:15.557597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f676bf-78f8-4acb-b8fd-bd0041f9da5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:15.558295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f676bf-78f8-4acb-b8fd-bd0041f9da5e admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:15.558577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f676bf-78f8-4acb-b8fd-bd0041f9da5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:15.558859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f676bf-78f8-4acb-b8fd-bd0041f9da5e admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:15.559617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f676bf-78f8-4acb-b8fd-bd0041f9da5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:15.561237 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f676bf-78f8-4acb-b8fd-bd0041f9da5e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:15.576060 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1144/2244] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:15.577631 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b777cb-a1b9-4b32-b4e9-9a0502ada87d admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:15.590277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b777cb-a1b9-4b32-b4e9-9a0502ada87d admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:15.595842 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a8a50-b1c3-4ef3-b5d6-ee9d33425b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:15.596971 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a8a50-b1c3-4ef3-b5d6-ee9d33425b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:15.597255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a8a50-b1c3-4ef3-b5d6-ee9d33425b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:15.597594 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a8a50-b1c3-4ef3-b5d6-ee9d33425b8f admin admin] PATH_INFO: `/v3/projects/6cc30d0af5914435810cfd1f94050758/users/5dcf7a9c81db4369b1e4607eed8ae927/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:15.616427 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a8a50-b1c3-4ef3-b5d6-ee9d33425b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc30d0af5914435810cfd1f94050758, user_id=5dcf7a9c81db4369b1e4607eed8ae927, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:15.619296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a8a50-b1c3-4ef3-b5d6-ee9d33425b8f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:15.633781 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1101/2245] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:15 2022] PUT /identity/v3/projects/6cc30d0af5914435810cfd1f94050758/users/5dcf7a9c81db4369b1e4607eed8ae927/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:15.637785 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f676bf-78f8-4acb-b8fd-bd0041f9da5e admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:15.649194 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f676bf-78f8-4acb-b8fd-bd0041f9da5e admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:15.712000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a141794-be0c-483c-8cb3-2e035a9a9fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:15.712673 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a141794-be0c-483c-8cb3-2e035a9a9fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:15.712990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a141794-be0c-483c-8cb3-2e035a9a9fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:15.713270 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a141794-be0c-483c-8cb3-2e035a9a9fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:15.713997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a141794-be0c-483c-8cb3-2e035a9a9fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:15.715612 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a141794-be0c-483c-8cb3-2e035a9a9fb2 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:15.726977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1145/2246] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:15.731374 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a8a50-b1c3-4ef3-b5d6-ee9d33425b8f admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:15.742002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a8a50-b1c3-4ef3-b5d6-ee9d33425b8f admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:15.746720 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85628e-0868-4c9f-a603-ebc8a8124694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:15.747603 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85628e-0868-4c9f-a603-ebc8a8124694 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:15.747922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85628e-0868-4c9f-a603-ebc8a8124694 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:15.748203 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85628e-0868-4c9f-a603-ebc8a8124694 admin admin] PATH_INFO: `/v3/projects/6cc30d0af5914435810cfd1f94050758/users/5dcf7a9c81db4369b1e4607eed8ae927/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:15.750182 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad85628e-0868-4c9f-a603-ebc8a8124694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc30d0af5914435810cfd1f94050758, user_id=5dcf7a9c81db4369b1e4607eed8ae927, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:15.752842 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad85628e-0868-4c9f-a603-ebc8a8124694 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:15.763438 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1102/2247] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:15 2022] PUT /identity/v3/projects/6cc30d0af5914435810cfd1f94050758/users/5dcf7a9c81db4369b1e4607eed8ae927/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:15.775104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729b9d9-7279-4f51-85cd-b1417e2694ca None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:15.775394 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729b9d9-7279-4f51-85cd-b1417e2694ca None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:15.775698 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729b9d9-7279-4f51-85cd-b1417e2694ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:15.803016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-1729b9d9-7279-4f51-85cd-b1417e2694ca None None] MFA Rules not processed for user `5dcf7a9c81db4369b1e4607eed8ae927`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:15.805289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1729b9d9-7279-4f51-85cd-b1417e2694ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:15.876771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1146/2248] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:15 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:15.888712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85628e-0868-4c9f-a603-ebc8a8124694 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:15.890444 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ad85628e-0868-4c9f-a603-ebc8a8124694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:15.954882 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85628e-0868-4c9f-a603-ebc8a8124694 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:15.958561 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea5aeb-3c28-44d9-9532-32cbd835d9a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:15.959208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea5aeb-3c28-44d9-9532-32cbd835d9a7 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:15.959492 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea5aeb-3c28-44d9-9532-32cbd835d9a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:15.959816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea5aeb-3c28-44d9-9532-32cbd835d9a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:15.970505 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea5aeb-3c28-44d9-9532-32cbd835d9a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:15.971973 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea5aeb-3c28-44d9-9532-32cbd835d9a7 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:15.982429 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1103/2249] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:15 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:16.077415 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729b9d9-7279-4f51-85cd-b1417e2694ca None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:16.078909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1729b9d9-7279-4f51-85cd-b1417e2694ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:16.091517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729b9d9-7279-4f51-85cd-b1417e2694ca None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:16.095278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a31e5-362d-4e4b-98d9-4cd941cf14d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:16.096031 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a31e5-362d-4e4b-98d9-4cd941cf14d9 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:16.096382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a31e5-362d-4e4b-98d9-4cd941cf14d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:16.096725 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a31e5-362d-4e4b-98d9-4cd941cf14d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:16.159573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a31e5-362d-4e4b-98d9-4cd941cf14d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:16.161106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a31e5-362d-4e4b-98d9-4cd941cf14d9 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:16.170441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1147/2250] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:23.819924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea5aeb-3c28-44d9-9532-32cbd835d9a7 service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:23.821456 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8aea5aeb-3c28-44d9-9532-32cbd835d9a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:23.918293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea5aeb-3c28-44d9-9532-32cbd835d9a7 service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:23.922874 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c0151-c7e8-4b97-aeed-e69c774004cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:23.923786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c0151-c7e8-4b97-aeed-e69c774004cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:23.924071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c0151-c7e8-4b97-aeed-e69c774004cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:23.924349 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c0151-c7e8-4b97-aeed-e69c774004cb admin admin] PATH_INFO: `/v3/users/26f0f6230a6b4f54b43e16ae12aa7a2e` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:23.925566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c0151-c7e8-4b97-aeed-e69c774004cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=26f0f6230a6b4f54b43e16ae12aa7a2e)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:23.927526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c0151-c7e8-4b97-aeed-e69c774004cb admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:23.941400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:23.975640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ce5c0151-c7e8-4b97-aeed-e69c774004cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26f0f6230a6b4f54b43e16ae12aa7a2e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:23.983387 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ce5c0151-c7e8-4b97-aeed-e69c774004cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26f0f6230a6b4f54b43e16ae12aa7a2e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:23.984011 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ce5c0151-c7e8-4b97-aeed-e69c774004cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26f0f6230a6b4f54b43e16ae12aa7a2e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:23.992462 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ce5c0151-c7e8-4b97-aeed-e69c774004cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26f0f6230a6b4f54b43e16ae12aa7a2e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:23.993223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ce5c0151-c7e8-4b97-aeed-e69c774004cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26f0f6230a6b4f54b43e16ae12aa7a2e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:24.012295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1104/2251] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:23 2022] DELETE /identity/v3/users/26f0f6230a6b4f54b43e16ae12aa7a2e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:24.016883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95132a9-d082-4c83-a8da-194f4405ae7a None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:24.017337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95132a9-d082-4c83-a8da-194f4405ae7a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:24.017451 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95132a9-d082-4c83-a8da-194f4405ae7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:24.029968 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:24.036690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e95132a9-d082-4c83-a8da-194f4405ae7a None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:24.037908 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e95132a9-d082-4c83-a8da-194f4405ae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:24.099137 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1148/2252] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:24.108929 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c0151-c7e8-4b97-aeed-e69c774004cb admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:24.110383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ce5c0151-c7e8-4b97-aeed-e69c774004cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:24.169922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c0151-c7e8-4b97-aeed-e69c774004cb admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:24.173564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ca9e8-e5cd-4239-ad37-98b28a09b309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:24.174217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ca9e8-e5cd-4239-ad37-98b28a09b309 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:24.174504 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ca9e8-e5cd-4239-ad37-98b28a09b309 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:24.174785 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ca9e8-e5cd-4239-ad37-98b28a09b309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:24.184137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ca9e8-e5cd-4239-ad37-98b28a09b309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:24.185600 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ca9e8-e5cd-4239-ad37-98b28a09b309 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:24.195836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1105/2253] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:24.489683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95132a9-d082-4c83-a8da-194f4405ae7a None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:24.492026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e95132a9-d082-4c83-a8da-194f4405ae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:24.508231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95132a9-d082-4c83-a8da-194f4405ae7a None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:24.513809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86197f89-7021-4a62-bdce-8ac187403fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:24.514750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86197f89-7021-4a62-bdce-8ac187403fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:24.515430 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86197f89-7021-4a62-bdce-8ac187403fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:24.515430 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86197f89-7021-4a62-bdce-8ac187403fa5 admin admin] PATH_INFO: `/v3/projects/ca3d3b69646549fdbdaf9377921bd610` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:24.516733 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86197f89-7021-4a62-bdce-8ac187403fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca3d3b69646549fdbdaf9377921bd610)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:24.518926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86197f89-7021-4a62-bdce-8ac187403fa5 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:24.569670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-86197f89-7021-4a62-bdce-8ac187403fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca3d3b69646549fdbdaf9377921bd610', 'request_id': 'req-86197f89-7021-4a62-bdce-8ac187403fa5'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:24.576314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-86197f89-7021-4a62-bdce-8ac187403fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca3d3b69646549fdbdaf9377921bd610', 'request_id': 'req-86197f89-7021-4a62-bdce-8ac187403fa5'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:24.582196 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-86197f89-7021-4a62-bdce-8ac187403fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca3d3b69646549fdbdaf9377921bd610', 'request_id': 'req-86197f89-7021-4a62-bdce-8ac187403fa5'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:24.595012 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-86197f89-7021-4a62-bdce-8ac187403fa5 admin admin] The token cache is being invalidate because project ca3d3b69646549fdbdaf9377921bd610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:24.595300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-86197f89-7021-4a62-bdce-8ac187403fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:24.595973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-86197f89-7021-4a62-bdce-8ac187403fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:24.597569 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1149/2254] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:24 2022] DELETE /identity/v3/projects/ca3d3b69646549fdbdaf9377921bd610 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:24.602373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ca9e8-e5cd-4239-ad37-98b28a09b309 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:24.612763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ca9e8-e5cd-4239-ad37-98b28a09b309 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:24.678870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105b970-3caa-45d8-9402-768f4e81b1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:24.680559 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105b970-3caa-45d8-9402-768f4e81b1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:24.680848 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105b970-3caa-45d8-9402-768f4e81b1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:24.681121 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105b970-3caa-45d8-9402-768f4e81b1fc admin admin] PATH_INFO: `/v3/users/ae1d737f538e4e85b861acb9fbd35542` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:24.682475 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105b970-3caa-45d8-9402-768f4e81b1fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae1d737f538e4e85b861acb9fbd35542)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:24.684868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105b970-3caa-45d8-9402-768f4e81b1fc admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:24.731053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:24.745700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2105b970-3caa-45d8-9402-768f4e81b1fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae1d737f538e4e85b861acb9fbd35542'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:24.757494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2105b970-3caa-45d8-9402-768f4e81b1fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae1d737f538e4e85b861acb9fbd35542'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:24.758448 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2105b970-3caa-45d8-9402-768f4e81b1fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae1d737f538e4e85b861acb9fbd35542'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:24.766838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2105b970-3caa-45d8-9402-768f4e81b1fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae1d737f538e4e85b861acb9fbd35542'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:24.767559 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2105b970-3caa-45d8-9402-768f4e81b1fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae1d737f538e4e85b861acb9fbd35542'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:24.787504 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1106/2255] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:24 2022] DELETE /identity/v3/users/ae1d737f538e4e85b861acb9fbd35542 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:54:25.317157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86197f89-7021-4a62-bdce-8ac187403fa5 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:25.319278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-86197f89-7021-4a62-bdce-8ac187403fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:25.367744 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86197f89-7021-4a62-bdce-8ac187403fa5 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:25.439868 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240111c-8945-4439-b95e-c97d7a1d3a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:25.440803 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240111c-8945-4439-b95e-c97d7a1d3a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:25.441150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240111c-8945-4439-b95e-c97d7a1d3a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:25.441487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240111c-8945-4439-b95e-c97d7a1d3a4d admin admin] PATH_INFO: `/v3/projects/2b46bf2b1b5e4bfd9f3817b8ad5c0110` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:25.442813 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240111c-8945-4439-b95e-c97d7a1d3a4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b46bf2b1b5e4bfd9f3817b8ad5c0110)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:25.446439 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240111c-8945-4439-b95e-c97d7a1d3a4d admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:25.489637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:25.505915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c240111c-8945-4439-b95e-c97d7a1d3a4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b46bf2b1b5e4bfd9f3817b8ad5c0110', 'request_id': 'req-c240111c-8945-4439-b95e-c97d7a1d3a4d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:25.513669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c240111c-8945-4439-b95e-c97d7a1d3a4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b46bf2b1b5e4bfd9f3817b8ad5c0110', 'request_id': 'req-c240111c-8945-4439-b95e-c97d7a1d3a4d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:25.520247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c240111c-8945-4439-b95e-c97d7a1d3a4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b46bf2b1b5e4bfd9f3817b8ad5c0110', 'request_id': 'req-c240111c-8945-4439-b95e-c97d7a1d3a4d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:25.532859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c240111c-8945-4439-b95e-c97d7a1d3a4d admin admin] The token cache is being invalidate because project 2b46bf2b1b5e4bfd9f3817b8ad5c0110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:25.533148 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c240111c-8945-4439-b95e-c97d7a1d3a4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:25.533949 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c240111c-8945-4439-b95e-c97d7a1d3a4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:25.535520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1150/2256] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:25 2022] DELETE /identity/v3/projects/2b46bf2b1b5e4bfd9f3817b8ad5c0110 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:25.540855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105b970-3caa-45d8-9402-768f4e81b1fc admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:25.542891 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-2105b970-3caa-45d8-9402-768f4e81b1fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:25.558918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105b970-3caa-45d8-9402-768f4e81b1fc admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:25.631454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7a9b6-65a7-45d9-8ec9-626c15e5049d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:25.632537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7a9b6-65a7-45d9-8ec9-626c15e5049d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:25.632707 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7a9b6-65a7-45d9-8ec9-626c15e5049d admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:25.632981 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7a9b6-65a7-45d9-8ec9-626c15e5049d admin admin] PATH_INFO: `/v3/users/0d52c7c081c2469b9ffeecbb408e4fae` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:25.634212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7a9b6-65a7-45d9-8ec9-626c15e5049d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d52c7c081c2469b9ffeecbb408e4fae)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:25.636194 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7a9b6-65a7-45d9-8ec9-626c15e5049d admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:25.685447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-16d7a9b6-65a7-45d9-8ec9-626c15e5049d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d52c7c081c2469b9ffeecbb408e4fae'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:25.692095 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240111c-8945-4439-b95e-c97d7a1d3a4d admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:25.693262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-16d7a9b6-65a7-45d9-8ec9-626c15e5049d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d52c7c081c2469b9ffeecbb408e4fae'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:25.693529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c240111c-8945-4439-b95e-c97d7a1d3a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:25.694037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-16d7a9b6-65a7-45d9-8ec9-626c15e5049d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d52c7c081c2469b9ffeecbb408e4fae'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:25.702167 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-16d7a9b6-65a7-45d9-8ec9-626c15e5049d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d52c7c081c2469b9ffeecbb408e4fae'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:25.702796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-16d7a9b6-65a7-45d9-8ec9-626c15e5049d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d52c7c081c2469b9ffeecbb408e4fae'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:25.706554 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240111c-8945-4439-b95e-c97d7a1d3a4d admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:25.712104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2d555-06ff-4ff9-b0ae-7ef849e777a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:25.712965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2d555-06ff-4ff9-b0ae-7ef849e777a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:25.713252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2d555-06ff-4ff9-b0ae-7ef849e777a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:25.713531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2d555-06ff-4ff9-b0ae-7ef849e777a4 admin admin] PATH_INFO: `/v3/users/420ecfdfc7f14d49a816d1506e8c4e7d` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:25.714754 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2d555-06ff-4ff9-b0ae-7ef849e777a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=420ecfdfc7f14d49a816d1506e8c4e7d)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:25.716719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2d555-06ff-4ff9-b0ae-7ef849e777a4 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:25.721243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1107/2257] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:25 2022] DELETE /identity/v3/users/0d52c7c081c2469b9ffeecbb408e4fae => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:54:25.767663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-28c2d555-06ff-4ff9-b0ae-7ef849e777a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '420ecfdfc7f14d49a816d1506e8c4e7d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:25.768361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-28c2d555-06ff-4ff9-b0ae-7ef849e777a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '420ecfdfc7f14d49a816d1506e8c4e7d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:25.782242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-28c2d555-06ff-4ff9-b0ae-7ef849e777a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '420ecfdfc7f14d49a816d1506e8c4e7d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:25.789142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-28c2d555-06ff-4ff9-b0ae-7ef849e777a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '420ecfdfc7f14d49a816d1506e8c4e7d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:25.797652 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-28c2d555-06ff-4ff9-b0ae-7ef849e777a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '420ecfdfc7f14d49a816d1506e8c4e7d'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:25.804595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1151/2258] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:25 2022] DELETE /identity/v3/users/420ecfdfc7f14d49a816d1506e8c4e7d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:54:25.809272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cfaf0-311b-48ba-94f0-1686281876a7 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:25.809559 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cfaf0-311b-48ba-94f0-1686281876a7 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:25.809834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cfaf0-311b-48ba-94f0-1686281876a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:25.827978 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-3d6cfaf0-311b-48ba-94f0-1686281876a7 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:25.829178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-3d6cfaf0-311b-48ba-94f0-1686281876a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:25.858132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:25.905280 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1108/2259] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:25.915352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2d555-06ff-4ff9-b0ae-7ef849e777a4 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:25.916925 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-28c2d555-06ff-4ff9-b0ae-7ef849e777a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:25.981726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2d555-06ff-4ff9-b0ae-7ef849e777a4 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:25.985257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41b979-8201-4087-868e-6e5c0ce98940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:25.986365 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41b979-8201-4087-868e-6e5c0ce98940 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:25.986572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41b979-8201-4087-868e-6e5c0ce98940 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:25.986852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41b979-8201-4087-868e-6e5c0ce98940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:25.997028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41b979-8201-4087-868e-6e5c0ce98940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:25.998559 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41b979-8201-4087-868e-6e5c0ce98940 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:26.003257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cfaf0-311b-48ba-94f0-1686281876a7 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:26.004910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-3d6cfaf0-311b-48ba-94f0-1686281876a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:26.012370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1152/2260] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:26.018596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cfaf0-311b-48ba-94f0-1686281876a7 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:26.024725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:26.025736 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:26.028131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:26.028404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e admin admin] PATH_INFO: `/v3/projects/690d4c05819c4a7591cc9c4f0fd12cd3` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:26.029642 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e admin admin] RBAC: Authorizing `identity:delete_project(project_id=690d4c05819c4a7591cc9c4f0fd12cd3)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:26.032351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:26.093483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '690d4c05819c4a7591cc9c4f0fd12cd3', 'request_id': 'req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:26.101848 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '690d4c05819c4a7591cc9c4f0fd12cd3', 'request_id': 'req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:26.108571 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '690d4c05819c4a7591cc9c4f0fd12cd3', 'request_id': 'req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:26.121294 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e admin admin] The token cache is being invalidate because project 690d4c05819c4a7591cc9c4f0fd12cd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:26.121584 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:26.122236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:26.124272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1109/2261] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:26 2022] DELETE /identity/v3/projects/690d4c05819c4a7591cc9c4f0fd12cd3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:54:26.368083 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41b979-8201-4087-868e-6e5c0ce98940 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:26.369585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-4d41b979-8201-4087-868e-6e5c0ce98940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:26.385116 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41b979-8201-4087-868e-6e5c0ce98940 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:26.436682 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:26.473925 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5380c90-03e0-444a-9327-6ba26c6460c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:26.474791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5380c90-03e0-444a-9327-6ba26c6460c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:26.475080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5380c90-03e0-444a-9327-6ba26c6460c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:26.475361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5380c90-03e0-444a-9327-6ba26c6460c4 admin admin] PATH_INFO: `/v3/projects/972647dccecd4053acc4cfb3b4d526f7` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:26.477309 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5380c90-03e0-444a-9327-6ba26c6460c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=972647dccecd4053acc4cfb3b4d526f7)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:26.479399 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5380c90-03e0-444a-9327-6ba26c6460c4 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:26.543393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d5380c90-03e0-444a-9327-6ba26c6460c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '972647dccecd4053acc4cfb3b4d526f7', 'request_id': 'req-d5380c90-03e0-444a-9327-6ba26c6460c4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:26.555319 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d5380c90-03e0-444a-9327-6ba26c6460c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '972647dccecd4053acc4cfb3b4d526f7', 'request_id': 'req-d5380c90-03e0-444a-9327-6ba26c6460c4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:26.559398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d5380c90-03e0-444a-9327-6ba26c6460c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '972647dccecd4053acc4cfb3b4d526f7', 'request_id': 'req-d5380c90-03e0-444a-9327-6ba26c6460c4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:26.578251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d5380c90-03e0-444a-9327-6ba26c6460c4 admin admin] The token cache is being invalidate because project 972647dccecd4053acc4cfb3b4d526f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:26.579072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d5380c90-03e0-444a-9327-6ba26c6460c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:26.579169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d5380c90-03e0-444a-9327-6ba26c6460c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:26.583608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1153/2262] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:26 2022] DELETE /identity/v3/projects/972647dccecd4053acc4cfb3b4d526f7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:26.585124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:26.587795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:26.602819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c1a16-53c5-4dd5-b9de-40d45c70171e admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:26.677453 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f2e6d-4741-448f-8703-63784909e77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:26.678343 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f2e6d-4741-448f-8703-63784909e77a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:26.679367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f2e6d-4741-448f-8703-63784909e77a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:26.679933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f2e6d-4741-448f-8703-63784909e77a admin admin] PATH_INFO: `/v3/users/21e6bacbf6014abfae46bdea300d4a29` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:26.681441 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f2e6d-4741-448f-8703-63784909e77a admin admin] RBAC: Authorizing `identity:delete_user(user_id=21e6bacbf6014abfae46bdea300d4a29)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:26.683415 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f2e6d-4741-448f-8703-63784909e77a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:26.737724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-c46f2e6d-4741-448f-8703-63784909e77a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21e6bacbf6014abfae46bdea300d4a29'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:26.745754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-c46f2e6d-4741-448f-8703-63784909e77a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21e6bacbf6014abfae46bdea300d4a29'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:26.746407 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-c46f2e6d-4741-448f-8703-63784909e77a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21e6bacbf6014abfae46bdea300d4a29'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:26.749045 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:26.757362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-c46f2e6d-4741-448f-8703-63784909e77a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21e6bacbf6014abfae46bdea300d4a29'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:26.758665 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-c46f2e6d-4741-448f-8703-63784909e77a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21e6bacbf6014abfae46bdea300d4a29'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:26.778000 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1110/2263] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:26 2022] DELETE /identity/v3/users/21e6bacbf6014abfae46bdea300d4a29 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:27.075701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5380c90-03e0-444a-9327-6ba26c6460c4 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:27.077228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d5380c90-03e0-444a-9327-6ba26c6460c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:27.091455 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5380c90-03e0-444a-9327-6ba26c6460c4 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:27.151319 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad1d75-8c6d-4b79-b02c-7ff43df00012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:27.152211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad1d75-8c6d-4b79-b02c-7ff43df00012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:27.152491 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad1d75-8c6d-4b79-b02c-7ff43df00012 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:27.152765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad1d75-8c6d-4b79-b02c-7ff43df00012 admin admin] PATH_INFO: `/v3/projects/ee0e855346524a838236dc89e2eccfe4` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:27.153982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad1d75-8c6d-4b79-b02c-7ff43df00012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee0e855346524a838236dc89e2eccfe4)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:27.156238 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad1d75-8c6d-4b79-b02c-7ff43df00012 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:27.204516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-11ad1d75-8c6d-4b79-b02c-7ff43df00012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee0e855346524a838236dc89e2eccfe4', 'request_id': 'req-11ad1d75-8c6d-4b79-b02c-7ff43df00012'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:27.214396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-11ad1d75-8c6d-4b79-b02c-7ff43df00012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee0e855346524a838236dc89e2eccfe4', 'request_id': 'req-11ad1d75-8c6d-4b79-b02c-7ff43df00012'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:27.220370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-11ad1d75-8c6d-4b79-b02c-7ff43df00012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee0e855346524a838236dc89e2eccfe4', 'request_id': 'req-11ad1d75-8c6d-4b79-b02c-7ff43df00012'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:27.224070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:27.237006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-11ad1d75-8c6d-4b79-b02c-7ff43df00012 admin admin] The token cache is being invalidate because project ee0e855346524a838236dc89e2eccfe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:27.237857 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-11ad1d75-8c6d-4b79-b02c-7ff43df00012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:27.238927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-11ad1d75-8c6d-4b79-b02c-7ff43df00012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:27.240521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1154/2264] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:27 2022] DELETE /identity/v3/projects/ee0e855346524a838236dc89e2eccfe4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:27.245815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45388804-bd9b-46da-8172-f1ebd845b431 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:27.246038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45388804-bd9b-46da-8172-f1ebd845b431 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:27.246322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45388804-bd9b-46da-8172-f1ebd845b431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:27.265519 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-45388804-bd9b-46da-8172-f1ebd845b431 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:27.266754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-45388804-bd9b-46da-8172-f1ebd845b431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:27.330030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1111/2265] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:27.338942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b440797-53a9-4994-b101-956a837fcd37 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:27.339228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b440797-53a9-4994-b101-956a837fcd37 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:27.339510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b440797-53a9-4994-b101-956a837fcd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:27.359057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-8b440797-53a9-4994-b101-956a837fcd37 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:27.360309 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8b440797-53a9-4994-b101-956a837fcd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:27.369661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1155/2266] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:27.375269 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45388804-bd9b-46da-8172-f1ebd845b431 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:27.385780 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45388804-bd9b-46da-8172-f1ebd845b431 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:27.391090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104a967-3594-4fcc-aa6d-df7a40ababd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:27.391914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104a967-3594-4fcc-aa6d-df7a40ababd6 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:27.392232 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104a967-3594-4fcc-aa6d-df7a40ababd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:27.392568 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104a967-3594-4fcc-aa6d-df7a40ababd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:27.393355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104a967-3594-4fcc-aa6d-df7a40ababd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:54:27.393767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104a967-3594-4fcc-aa6d-df7a40ababd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:27.395621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104a967-3594-4fcc-aa6d-df7a40ababd6 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:27.411744 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1112/2267] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:54:27.414186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b440797-53a9-4994-b101-956a837fcd37 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:27.423863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b440797-53a9-4994-b101-956a837fcd37 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:27.428998 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18a85e-a53c-4cc0-b594-cdacfaefef0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:27.429801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18a85e-a53c-4cc0-b594-cdacfaefef0f admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:27.430082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18a85e-a53c-4cc0-b594-cdacfaefef0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:27.430362 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18a85e-a53c-4cc0-b594-cdacfaefef0f admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:27.431063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18a85e-a53c-4cc0-b594-cdacfaefef0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:27.433028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18a85e-a53c-4cc0-b594-cdacfaefef0f admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:27.469483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1156/2268] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:27 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:54:27.472203 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104a967-3594-4fcc-aa6d-df7a40ababd6 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:27.483475 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104a967-3594-4fcc-aa6d-df7a40ababd6 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:27.559170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03ae54-ee11-42f9-90d0-0e3f1b610810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:27.559902 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03ae54-ee11-42f9-90d0-0e3f1b610810 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:27.560192 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03ae54-ee11-42f9-90d0-0e3f1b610810 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:27.560468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03ae54-ee11-42f9-90d0-0e3f1b610810 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:27.561182 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03ae54-ee11-42f9-90d0-0e3f1b610810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:27.563118 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03ae54-ee11-42f9-90d0-0e3f1b610810 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:27.589772 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1113/2269] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:27 2022] POST /identity/v3/users => generated 285 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:27.596118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18a85e-a53c-4cc0-b594-cdacfaefef0f admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:27.614992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18a85e-a53c-4cc0-b594-cdacfaefef0f admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:27.621578 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146f74f-cd87-4322-9a39-2755caad3304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:27.622261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146f74f-cd87-4322-9a39-2755caad3304 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:27.622624 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146f74f-cd87-4322-9a39-2755caad3304 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:27.622822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146f74f-cd87-4322-9a39-2755caad3304 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:27.623574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146f74f-cd87-4322-9a39-2755caad3304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:27.625259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146f74f-cd87-4322-9a39-2755caad3304 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:27.639757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1157/2270] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:27.641171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03ae54-ee11-42f9-90d0-0e3f1b610810 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:27.653486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03ae54-ee11-42f9-90d0-0e3f1b610810 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:27.659040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ffe5c8-449f-4075-bdee-5c26a8dd625c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:27.659740 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffe5c8-449f-4075-bdee-5c26a8dd625c admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:27.660068 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffe5c8-449f-4075-bdee-5c26a8dd625c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:27.660344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffe5c8-449f-4075-bdee-5c26a8dd625c admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:27.661107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ffe5c8-449f-4075-bdee-5c26a8dd625c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:27.662881 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ffe5c8-449f-4075-bdee-5c26a8dd625c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:27.685008 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1114/2271] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:27.691922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146f74f-cd87-4322-9a39-2755caad3304 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:27.703363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146f74f-cd87-4322-9a39-2755caad3304 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:27.708073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4e053-aafe-485a-bbea-9dec593cb5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:27.708965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4e053-aafe-485a-bbea-9dec593cb5ca admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:27.709243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4e053-aafe-485a-bbea-9dec593cb5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:27.709512 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4e053-aafe-485a-bbea-9dec593cb5ca admin admin] PATH_INFO: `/v3/projects/b7497b0c2c9c42dda38332bb368e56bb/users/ec5dffdb634746548049eb6caa707d74/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:27.727249 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4e053-aafe-485a-bbea-9dec593cb5ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7497b0c2c9c42dda38332bb368e56bb, user_id=ec5dffdb634746548049eb6caa707d74, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:27.730494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4e053-aafe-485a-bbea-9dec593cb5ca admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:27.755840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1158/2272] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:27 2022] PUT /identity/v3/projects/b7497b0c2c9c42dda38332bb368e56bb/users/ec5dffdb634746548049eb6caa707d74/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:27.758686 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191104b1-9590-48f8-b3c6-272c6455aca2 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:27.758980 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191104b1-9590-48f8-b3c6-272c6455aca2 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:27.759261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191104b1-9590-48f8-b3c6-272c6455aca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:27.790512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-191104b1-9590-48f8-b3c6-272c6455aca2 None None] MFA Rules not processed for user `ec5dffdb634746548049eb6caa707d74`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:27.791771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-191104b1-9590-48f8-b3c6-272c6455aca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:27.824910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:27.862928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1115/2273] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:27.875908 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4e053-aafe-485a-bbea-9dec593cb5ca admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:27.877658 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ddd4e053-aafe-485a-bbea-9dec593cb5ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:27.933585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4e053-aafe-485a-bbea-9dec593cb5ca admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:27.937228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb7b2f-f113-4aee-a733-c7eaaac6b4e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d596de177e24ca5a254bda6644a07da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:27.937866 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb7b2f-f113-4aee-a733-c7eaaac6b4e0 service glance] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:27.938143 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb7b2f-f113-4aee-a733-c7eaaac6b4e0 service glance] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:27.938416 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb7b2f-f113-4aee-a733-c7eaaac6b4e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:27.947500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb7b2f-f113-4aee-a733-c7eaaac6b4e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:27.948963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb7b2f-f113-4aee-a733-c7eaaac6b4e0 service glance] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:27.958026 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1159/2274] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:27 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:30.017997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191104b1-9590-48f8-b3c6-272c6455aca2 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:30.019624 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-191104b1-9590-48f8-b3c6-272c6455aca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:30.096001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191104b1-9590-48f8-b3c6-272c6455aca2 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:30.100514 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4c79b-16bc-4822-9fdd-49c70e321947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:30.101512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4c79b-16bc-4822-9fdd-49c70e321947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:30.101793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4c79b-16bc-4822-9fdd-49c70e321947 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:30.102068 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4c79b-16bc-4822-9fdd-49c70e321947 admin admin] PATH_INFO: `/v3/users/de1399f02bdf42c6980a850424cb4019` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:30.103267 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4c79b-16bc-4822-9fdd-49c70e321947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de1399f02bdf42c6980a850424cb4019)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:30.105217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4c79b-16bc-4822-9fdd-49c70e321947 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:30.156537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-39b4c79b-16bc-4822-9fdd-49c70e321947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de1399f02bdf42c6980a850424cb4019'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:30.165563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-39b4c79b-16bc-4822-9fdd-49c70e321947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de1399f02bdf42c6980a850424cb4019'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:30.166179 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-39b4c79b-16bc-4822-9fdd-49c70e321947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de1399f02bdf42c6980a850424cb4019'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:30.182165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-39b4c79b-16bc-4822-9fdd-49c70e321947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de1399f02bdf42c6980a850424cb4019'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:30.182896 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-39b4c79b-16bc-4822-9fdd-49c70e321947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de1399f02bdf42c6980a850424cb4019'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:30.202589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1116/2275] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:30 2022] DELETE /identity/v3/users/de1399f02bdf42c6980a850424cb4019 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:30.207215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1436970-9e49-493d-b440-63a9d7054b45 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:30.207694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1436970-9e49-493d-b440-63a9d7054b45 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:30.207838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1436970-9e49-493d-b440-63a9d7054b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:30.227001 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-b1436970-9e49-493d-b440-63a9d7054b45 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:30.228315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b1436970-9e49-493d-b440-63a9d7054b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:30.290212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1160/2276] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:54:30.300290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4c79b-16bc-4822-9fdd-49c70e321947 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:30.302031 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-39b4c79b-16bc-4822-9fdd-49c70e321947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:30.357545 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:30.362183 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4c79b-16bc-4822-9fdd-49c70e321947 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:30.368767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c74eda-ce2c-4ece-8695-317782d7fcd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:30.370009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c74eda-ce2c-4ece-8695-317782d7fcd8 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:30.370330 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c74eda-ce2c-4ece-8695-317782d7fcd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:30.370863 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c74eda-ce2c-4ece-8695-317782d7fcd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:30.384627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c74eda-ce2c-4ece-8695-317782d7fcd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:30.386246 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c74eda-ce2c-4ece-8695-317782d7fcd8 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:30.398777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1117/2277] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:30.695821 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1436970-9e49-493d-b440-63a9d7054b45 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:30.697686 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b1436970-9e49-493d-b440-63a9d7054b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:30.714452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1436970-9e49-493d-b440-63a9d7054b45 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:30.720646 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:30.721540 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:30.721829 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:30.722110 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5 admin admin] PATH_INFO: `/v3/projects/de0fa93dd8ed4ff094790c90d27faf51` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:30.724074 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de0fa93dd8ed4ff094790c90d27faf51)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:30.727693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:30.776462 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:30.793561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de0fa93dd8ed4ff094790c90d27faf51', 'request_id': 'req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:30.800776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de0fa93dd8ed4ff094790c90d27faf51', 'request_id': 'req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:30.808001 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de0fa93dd8ed4ff094790c90d27faf51', 'request_id': 'req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:30.821006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5 admin admin] The token cache is being invalidate because project de0fa93dd8ed4ff094790c90d27faf51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:30.821294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:30.822301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:30.823907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1161/2278] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:30 2022] DELETE /identity/v3/projects/de0fa93dd8ed4ff094790c90d27faf51 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:30.828227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c74eda-ce2c-4ece-8695-317782d7fcd8 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:30.838922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c74eda-ce2c-4ece-8695-317782d7fcd8 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:30.902807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370fc8b-5d5c-496b-87e2-f8c2ae06152e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:30.903702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370fc8b-5d5c-496b-87e2-f8c2ae06152e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:30.904018 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370fc8b-5d5c-496b-87e2-f8c2ae06152e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:30.904296 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370fc8b-5d5c-496b-87e2-f8c2ae06152e admin admin] PATH_INFO: `/v3/users/5dcf7a9c81db4369b1e4607eed8ae927` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:30.905532 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370fc8b-5d5c-496b-87e2-f8c2ae06152e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dcf7a9c81db4369b1e4607eed8ae927)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:30.907468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370fc8b-5d5c-496b-87e2-f8c2ae06152e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:30.961386 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d370fc8b-5d5c-496b-87e2-f8c2ae06152e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dcf7a9c81db4369b1e4607eed8ae927'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:30.969383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d370fc8b-5d5c-496b-87e2-f8c2ae06152e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dcf7a9c81db4369b1e4607eed8ae927'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:30.970013 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d370fc8b-5d5c-496b-87e2-f8c2ae06152e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dcf7a9c81db4369b1e4607eed8ae927'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:30.978226 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d370fc8b-5d5c-496b-87e2-f8c2ae06152e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dcf7a9c81db4369b1e4607eed8ae927'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:30.978963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d370fc8b-5d5c-496b-87e2-f8c2ae06152e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dcf7a9c81db4369b1e4607eed8ae927'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:31.000047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1118/2279] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:30 2022] DELETE /identity/v3/users/5dcf7a9c81db4369b1e4607eed8ae927 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:31.348263 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:31.349807 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:31.363980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8fe1b-8893-4a8b-bbff-3bc0bacbe9d5 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:31.431603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc149c7-dcf0-40f9-ab67-98444164bb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:31.432522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc149c7-dcf0-40f9-ab67-98444164bb10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:31.432806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc149c7-dcf0-40f9-ab67-98444164bb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:31.433082 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc149c7-dcf0-40f9-ab67-98444164bb10 admin admin] PATH_INFO: `/v3/projects/6cc30d0af5914435810cfd1f94050758` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:31.434384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc149c7-dcf0-40f9-ab67-98444164bb10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc30d0af5914435810cfd1f94050758)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:31.436287 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc149c7-dcf0-40f9-ab67-98444164bb10 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:31.483454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-afc149c7-dcf0-40f9-ab67-98444164bb10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc30d0af5914435810cfd1f94050758', 'request_id': 'req-afc149c7-dcf0-40f9-ab67-98444164bb10'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:31.489938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-afc149c7-dcf0-40f9-ab67-98444164bb10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc30d0af5914435810cfd1f94050758', 'request_id': 'req-afc149c7-dcf0-40f9-ab67-98444164bb10'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:31.497705 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-afc149c7-dcf0-40f9-ab67-98444164bb10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc30d0af5914435810cfd1f94050758', 'request_id': 'req-afc149c7-dcf0-40f9-ab67-98444164bb10'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:31.511506 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-afc149c7-dcf0-40f9-ab67-98444164bb10 admin admin] The token cache is being invalidate because project 6cc30d0af5914435810cfd1f94050758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:31.511827 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-afc149c7-dcf0-40f9-ab67-98444164bb10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:31.512457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-afc149c7-dcf0-40f9-ab67-98444164bb10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:31.514046 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1162/2280] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:31 2022] DELETE /identity/v3/projects/6cc30d0af5914435810cfd1f94050758 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:31.518896 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6820a32-cdaa-4c01-9749-d7df02ffe386 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:31.519347 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6820a32-cdaa-4c01-9749-d7df02ffe386 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:31.519451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6820a32-cdaa-4c01-9749-d7df02ffe386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:31.539341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-f6820a32-cdaa-4c01-9749-d7df02ffe386 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:31.540613 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f6820a32-cdaa-4c01-9749-d7df02ffe386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:31.611648 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1119/2281] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:31.620086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a785264-1090-4709-ae4e-44408e3d0b81 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:31.620372 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a785264-1090-4709-ae4e-44408e3d0b81 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:31.620650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a785264-1090-4709-ae4e-44408e3d0b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:31.639651 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-3a785264-1090-4709-ae4e-44408e3d0b81 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:31.640893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-3a785264-1090-4709-ae4e-44408e3d0b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:31.649408 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1163/2282] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:31.653828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6820a32-cdaa-4c01-9749-d7df02ffe386 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:31.664802 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6820a32-cdaa-4c01-9749-d7df02ffe386 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:31.669452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a19a1-1436-40c2-b5fa-c0134e85de87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:31.670240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a19a1-1436-40c2-b5fa-c0134e85de87 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:31.670556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a19a1-1436-40c2-b5fa-c0134e85de87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:31.670826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a19a1-1436-40c2-b5fa-c0134e85de87 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:31.671561 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a19a1-1436-40c2-b5fa-c0134e85de87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:54:31.671897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a19a1-1436-40c2-b5fa-c0134e85de87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:31.673497 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a19a1-1436-40c2-b5fa-c0134e85de87 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:31.680621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:31.687347 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1120/2283] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:54:31.692226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a785264-1090-4709-ae4e-44408e3d0b81 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:31.701743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a785264-1090-4709-ae4e-44408e3d0b81 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:31.706465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935c411-e95c-41dc-9b1b-afd50c39c781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:31.707113 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935c411-e95c-41dc-9b1b-afd50c39c781 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:31.707400 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935c411-e95c-41dc-9b1b-afd50c39c781 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:31.707678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935c411-e95c-41dc-9b1b-afd50c39c781 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:31.708409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935c411-e95c-41dc-9b1b-afd50c39c781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:31.710830 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935c411-e95c-41dc-9b1b-afd50c39c781 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:31.737160 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1164/2284] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:31 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:31.741364 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a19a1-1436-40c2-b5fa-c0134e85de87 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:31.751564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a19a1-1436-40c2-b5fa-c0134e85de87 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:31.811408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e7f5c-bf8c-415d-8ede-6994d2619e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:31.812117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e7f5c-bf8c-415d-8ede-6994d2619e30 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:31.812404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e7f5c-bf8c-415d-8ede-6994d2619e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:31.812694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e7f5c-bf8c-415d-8ede-6994d2619e30 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:31.813393 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e7f5c-bf8c-415d-8ede-6994d2619e30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:31.815317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e7f5c-bf8c-415d-8ede-6994d2619e30 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:31.835512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1121/2285] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:31 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:31.839685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935c411-e95c-41dc-9b1b-afd50c39c781 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:31.850895 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935c411-e95c-41dc-9b1b-afd50c39c781 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:31.855882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fe951-bf33-426b-9ab8-d8927d6653bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:31.856601 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fe951-bf33-426b-9ab8-d8927d6653bb admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:31.856922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fe951-bf33-426b-9ab8-d8927d6653bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:31.857202 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fe951-bf33-426b-9ab8-d8927d6653bb admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:31.857953 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05fe951-bf33-426b-9ab8-d8927d6653bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:31.859597 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05fe951-bf33-426b-9ab8-d8927d6653bb admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:31.873150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1165/2286] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:54:31.877218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e7f5c-bf8c-415d-8ede-6994d2619e30 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:31.887964 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e7f5c-bf8c-415d-8ede-6994d2619e30 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:31.892901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa3168-4a3e-44e3-a7df-e71dcfa69394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:31.893575 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa3168-4a3e-44e3-a7df-e71dcfa69394 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:31.893855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa3168-4a3e-44e3-a7df-e71dcfa69394 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:31.894122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa3168-4a3e-44e3-a7df-e71dcfa69394 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:31.894888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa3168-4a3e-44e3-a7df-e71dcfa69394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:31.896543 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa3168-4a3e-44e3-a7df-e71dcfa69394 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:31.907949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1122/2287] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:31.912499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fe951-bf33-426b-9ab8-d8927d6653bb admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:31.923666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fe951-bf33-426b-9ab8-d8927d6653bb admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:31.928608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76054748-d705-4619-8d92-999cb232e289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:31.930038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76054748-d705-4619-8d92-999cb232e289 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:31.930386 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76054748-d705-4619-8d92-999cb232e289 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:31.930660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76054748-d705-4619-8d92-999cb232e289 admin admin] PATH_INFO: `/v3/projects/91082d5e6cc94246b34b31b7518fb8d1/users/a146c0d3471b4feab7a02d5e0a1e2bde/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:31.934483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:31.949790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76054748-d705-4619-8d92-999cb232e289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91082d5e6cc94246b34b31b7518fb8d1, user_id=a146c0d3471b4feab7a02d5e0a1e2bde, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:31.952932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76054748-d705-4619-8d92-999cb232e289 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:31.966157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1166/2288] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:31 2022] PUT /identity/v3/projects/91082d5e6cc94246b34b31b7518fb8d1/users/a146c0d3471b4feab7a02d5e0a1e2bde/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:54:31.971729 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399b5a0-6620-4fe2-a726-e700ee48d585 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:31.972035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399b5a0-6620-4fe2-a726-e700ee48d585 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:31.972342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399b5a0-6620-4fe2-a726-e700ee48d585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:31.991021 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-9399b5a0-6620-4fe2-a726-e700ee48d585 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:31.993039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-9399b5a0-6620-4fe2-a726-e700ee48d585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:32.055092 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1123/2289] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:32.065854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76054748-d705-4619-8d92-999cb232e289 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:32.067639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-76054748-d705-4619-8d92-999cb232e289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:32.128138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76054748-d705-4619-8d92-999cb232e289 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:32.132648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febaba0-46b8-4186-96e9-fb78b14157f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:32.133382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febaba0-46b8-4186-96e9-fb78b14157f8 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:32.133672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febaba0-46b8-4186-96e9-fb78b14157f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:32.133948 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febaba0-46b8-4186-96e9-fb78b14157f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:32.143980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febaba0-46b8-4186-96e9-fb78b14157f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:32.145441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febaba0-46b8-4186-96e9-fb78b14157f8 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:32.155803 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1167/2290] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:32.895844 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a48021-442f-4cdc-b1d5-4ab531238817 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:32.896348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a48021-442f-4cdc-b1d5-4ab531238817 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:32.898030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a48021-442f-4cdc-b1d5-4ab531238817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:32.928318 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-92a48021-442f-4cdc-b1d5-4ab531238817 None None] MFA Rules not processed for user `a146c0d3471b4feab7a02d5e0a1e2bde`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:32.930291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-92a48021-442f-4cdc-b1d5-4ab531238817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:33.001793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1124/2291] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:33.006266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febaba0-46b8-4186-96e9-fb78b14157f8 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:33.016150 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febaba0-46b8-4186-96e9-fb78b14157f8 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:33.020721 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4564b1-2f20-40f3-8978-1f3553f37e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:33.021464 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4564b1-2f20-40f3-8978-1f3553f37e40 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:33.021820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4564b1-2f20-40f3-8978-1f3553f37e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:33.022167 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4564b1-2f20-40f3-8978-1f3553f37e40 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:33.023707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4564b1-2f20-40f3-8978-1f3553f37e40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:33.025724 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4564b1-2f20-40f3-8978-1f3553f37e40 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:33.056316 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1168/2292] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:33 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:33.057672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a48021-442f-4cdc-b1d5-4ab531238817 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:33.068351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a48021-442f-4cdc-b1d5-4ab531238817 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:33.072947 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:33.127794 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08154781-2fa1-4e32-bf15-a2f88a7ab2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:33.128470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08154781-2fa1-4e32-bf15-a2f88a7ab2fc admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:33.128756 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08154781-2fa1-4e32-bf15-a2f88a7ab2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:33.129033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08154781-2fa1-4e32-bf15-a2f88a7ab2fc admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:33.129731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08154781-2fa1-4e32-bf15-a2f88a7ab2fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:33.131769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08154781-2fa1-4e32-bf15-a2f88a7ab2fc admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:33.152512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1125/2293] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:33 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:33.156381 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4564b1-2f20-40f3-8978-1f3553f37e40 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:33.167108 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4564b1-2f20-40f3-8978-1f3553f37e40 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:33.171811 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87fde7-4da8-4e8b-9ce0-bb7d2f163177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:33.172660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87fde7-4da8-4e8b-9ce0-bb7d2f163177 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:33.172953 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87fde7-4da8-4e8b-9ce0-bb7d2f163177 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:33.173262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87fde7-4da8-4e8b-9ce0-bb7d2f163177 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:33.173995 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87fde7-4da8-4e8b-9ce0-bb7d2f163177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:33.175590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87fde7-4da8-4e8b-9ce0-bb7d2f163177 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:33.186809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1169/2294] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:54:33.190982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08154781-2fa1-4e32-bf15-a2f88a7ab2fc admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:33.202185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08154781-2fa1-4e32-bf15-a2f88a7ab2fc admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:33.208531 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ba220-2fc9-4a7c-a720-92bae1683eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:33.209455 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ba220-2fc9-4a7c-a720-92bae1683eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:33.210233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ba220-2fc9-4a7c-a720-92bae1683eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:33.210233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ba220-2fc9-4a7c-a720-92bae1683eb3 admin admin] PATH_INFO: `/v3/projects/b03b888d646c47afa4da0a23fca675eb/users/5ebfb2db400246048eb5107677aeee25/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:33.227579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ba220-2fc9-4a7c-a720-92bae1683eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b03b888d646c47afa4da0a23fca675eb, user_id=5ebfb2db400246048eb5107677aeee25, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:33.230430 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ba220-2fc9-4a7c-a720-92bae1683eb3 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:33.246817 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1126/2295] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:33 2022] PUT /identity/v3/projects/b03b888d646c47afa4da0a23fca675eb/users/5ebfb2db400246048eb5107677aeee25/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:33.250794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87fde7-4da8-4e8b-9ce0-bb7d2f163177 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:33.261578 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87fde7-4da8-4e8b-9ce0-bb7d2f163177 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:33.325414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e356e-6a8b-42de-9d2e-5280afb567eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:33.326117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e356e-6a8b-42de-9d2e-5280afb567eb admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:33.326409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e356e-6a8b-42de-9d2e-5280afb567eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:33.326691 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e356e-6a8b-42de-9d2e-5280afb567eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:33.327446 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e356e-6a8b-42de-9d2e-5280afb567eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:33.329153 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e356e-6a8b-42de-9d2e-5280afb567eb admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:33.343773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1170/2296] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:33.345425 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ba220-2fc9-4a7c-a720-92bae1683eb3 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:33.357582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ba220-2fc9-4a7c-a720-92bae1683eb3 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:33.363033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdce40-779d-4635-9438-bf81824978b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:33.364146 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdce40-779d-4635-9438-bf81824978b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:33.364666 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdce40-779d-4635-9438-bf81824978b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:33.365053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdce40-779d-4635-9438-bf81824978b3 admin admin] PATH_INFO: `/v3/projects/b03b888d646c47afa4da0a23fca675eb/users/5ebfb2db400246048eb5107677aeee25/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:33.368306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfdce40-779d-4635-9438-bf81824978b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b03b888d646c47afa4da0a23fca675eb, user_id=5ebfb2db400246048eb5107677aeee25, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:33.371222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfdce40-779d-4635-9438-bf81824978b3 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:33.384833 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1127/2297] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:33 2022] PUT /identity/v3/projects/b03b888d646c47afa4da0a23fca675eb/users/5ebfb2db400246048eb5107677aeee25/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:34.228309 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876c83a-b023-461b-b2e0-621ba7ff1602 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:34.228804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876c83a-b023-461b-b2e0-621ba7ff1602 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:34.228920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876c83a-b023-461b-b2e0-621ba7ff1602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:34.258730 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e876c83a-b023-461b-b2e0-621ba7ff1602 None None] MFA Rules not processed for user `5ebfb2db400246048eb5107677aeee25`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:34.260123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e876c83a-b023-461b-b2e0-621ba7ff1602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:34.311866 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:34.345078 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1171/2298] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:34.358360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdce40-779d-4635-9438-bf81824978b3 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:34.360826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8bfdce40-779d-4635-9438-bf81824978b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:34.433219 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdce40-779d-4635-9438-bf81824978b3 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:34.436976 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79526652-8615-48e0-ad08-31e62136b77c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:34.437553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79526652-8615-48e0-ad08-31e62136b77c service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:34.437992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79526652-8615-48e0-ad08-31e62136b77c service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:34.438279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79526652-8615-48e0-ad08-31e62136b77c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:34.508788 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79526652-8615-48e0-ad08-31e62136b77c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:34.510498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79526652-8615-48e0-ad08-31e62136b77c service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:34.520674 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1128/2299] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:38.025178 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876c83a-b023-461b-b2e0-621ba7ff1602 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:38.036450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876c83a-b023-461b-b2e0-621ba7ff1602 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:38.040894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d3f02-a3e2-49a5-b420-c089740a25ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:38.041545 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d3f02-a3e2-49a5-b420-c089740a25ad service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:38.041823 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d3f02-a3e2-49a5-b420-c089740a25ad service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:38.042092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d3f02-a3e2-49a5-b420-c089740a25ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:38.055551 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d3f02-a3e2-49a5-b420-c089740a25ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:38.057159 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d3f02-a3e2-49a5-b420-c089740a25ad service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:38.068280 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1172/2300] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:42.393486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79526652-8615-48e0-ad08-31e62136b77c service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:42.394929 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-79526652-8615-48e0-ad08-31e62136b77c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:42.422945 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:42.480963 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79526652-8615-48e0-ad08-31e62136b77c service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:42.485650 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a56b67-1ae2-46b2-8f5c-efe1482f5898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:42.486787 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a56b67-1ae2-46b2-8f5c-efe1482f5898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:42.487063 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a56b67-1ae2-46b2-8f5c-efe1482f5898 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:42.487335 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a56b67-1ae2-46b2-8f5c-efe1482f5898 admin admin] PATH_INFO: `/v3/users/5da6fa5f17b741ff86ca357cb5a17d7a` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:42.488591 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a56b67-1ae2-46b2-8f5c-efe1482f5898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5da6fa5f17b741ff86ca357cb5a17d7a)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:42.490727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a56b67-1ae2-46b2-8f5c-efe1482f5898 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:42.544957 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-24a56b67-1ae2-46b2-8f5c-efe1482f5898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5da6fa5f17b741ff86ca357cb5a17d7a'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:42.552645 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-24a56b67-1ae2-46b2-8f5c-efe1482f5898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5da6fa5f17b741ff86ca357cb5a17d7a'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:42.554886 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-24a56b67-1ae2-46b2-8f5c-efe1482f5898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5da6fa5f17b741ff86ca357cb5a17d7a'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:42.562769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-24a56b67-1ae2-46b2-8f5c-efe1482f5898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5da6fa5f17b741ff86ca357cb5a17d7a'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:42.564120 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-24a56b67-1ae2-46b2-8f5c-efe1482f5898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5da6fa5f17b741ff86ca357cb5a17d7a'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:42.585412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1129/2301] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:42 2022] DELETE /identity/v3/users/5da6fa5f17b741ff86ca357cb5a17d7a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:42.589987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae830a6-d7d5-4847-88a2-33dd6dc13a1f None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:42.590456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae830a6-d7d5-4847-88a2-33dd6dc13a1f None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:42.590833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae830a6-d7d5-4847-88a2-33dd6dc13a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:42.609853 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-aae830a6-d7d5-4847-88a2-33dd6dc13a1f None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:42.611332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-aae830a6-d7d5-4847-88a2-33dd6dc13a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:42.683348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1173/2302] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:54:42.689887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a56b67-1ae2-46b2-8f5c-efe1482f5898 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:42.691609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-24a56b67-1ae2-46b2-8f5c-efe1482f5898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:42.764399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a56b67-1ae2-46b2-8f5c-efe1482f5898 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:42.768107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb9c71-3cb0-48df-9646-ea477ef8bdab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:42.768759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb9c71-3cb0-48df-9646-ea477ef8bdab service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:42.769046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb9c71-3cb0-48df-9646-ea477ef8bdab service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:42.769325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb9c71-3cb0-48df-9646-ea477ef8bdab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:42.779243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb9c71-3cb0-48df-9646-ea477ef8bdab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:42.780720 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb9c71-3cb0-48df-9646-ea477ef8bdab service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:42.791383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1130/2303] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:43.366983 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae830a6-d7d5-4847-88a2-33dd6dc13a1f None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:43.368643 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-aae830a6-d7d5-4847-88a2-33dd6dc13a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:43.383187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae830a6-d7d5-4847-88a2-33dd6dc13a1f None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:43.401562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497a9b5-7f7f-404b-89c7-2c7f46841b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:43.406170 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497a9b5-7f7f-404b-89c7-2c7f46841b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:43.406520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497a9b5-7f7f-404b-89c7-2c7f46841b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:43.406879 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497a9b5-7f7f-404b-89c7-2c7f46841b93 admin admin] PATH_INFO: `/v3/projects/71b5f5e3101f4501b84fe6eba9697b19` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:43.408312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497a9b5-7f7f-404b-89c7-2c7f46841b93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b5f5e3101f4501b84fe6eba9697b19)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:43.410955 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497a9b5-7f7f-404b-89c7-2c7f46841b93 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:43.464934 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-f497a9b5-7f7f-404b-89c7-2c7f46841b93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b5f5e3101f4501b84fe6eba9697b19', 'request_id': 'req-f497a9b5-7f7f-404b-89c7-2c7f46841b93'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:43.471742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-f497a9b5-7f7f-404b-89c7-2c7f46841b93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b5f5e3101f4501b84fe6eba9697b19', 'request_id': 'req-f497a9b5-7f7f-404b-89c7-2c7f46841b93'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:43.480214 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-f497a9b5-7f7f-404b-89c7-2c7f46841b93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b5f5e3101f4501b84fe6eba9697b19', 'request_id': 'req-f497a9b5-7f7f-404b-89c7-2c7f46841b93'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:43.494097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-f497a9b5-7f7f-404b-89c7-2c7f46841b93 admin admin] The token cache is being invalidate because project 71b5f5e3101f4501b84fe6eba9697b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:43.494499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-f497a9b5-7f7f-404b-89c7-2c7f46841b93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:43.495282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-f497a9b5-7f7f-404b-89c7-2c7f46841b93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:43.496955 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1174/2304] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:43 2022] DELETE /identity/v3/projects/71b5f5e3101f4501b84fe6eba9697b19 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:43.501903 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80864c15-1670-4b01-bd35-c32e823f363e None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:43.502360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80864c15-1670-4b01-bd35-c32e823f363e None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:43.502582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80864c15-1670-4b01-bd35-c32e823f363e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:43.520304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-80864c15-1670-4b01-bd35-c32e823f363e None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:43.521622 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-80864c15-1670-4b01-bd35-c32e823f363e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:43.557468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:43.596200 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1131/2305] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:43.605445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebad8b0-9e2e-45a1-8432-5b556ac9c545 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:43.606030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebad8b0-9e2e-45a1-8432-5b556ac9c545 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:43.606144 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebad8b0-9e2e-45a1-8432-5b556ac9c545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:43.639822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-8ebad8b0-9e2e-45a1-8432-5b556ac9c545 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:43.642526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8ebad8b0-9e2e-45a1-8432-5b556ac9c545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:43.659583 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1175/2306] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:54:43.661945 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80864c15-1670-4b01-bd35-c32e823f363e None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:43.678142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80864c15-1670-4b01-bd35-c32e823f363e None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:43.686757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21248c3b-719b-4c3d-8dc9-86307b23d74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:43.687571 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21248c3b-719b-4c3d-8dc9-86307b23d74f admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:43.687961 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21248c3b-719b-4c3d-8dc9-86307b23d74f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:43.688306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21248c3b-719b-4c3d-8dc9-86307b23d74f admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:43.689768 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21248c3b-719b-4c3d-8dc9-86307b23d74f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:54:43.689768 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21248c3b-719b-4c3d-8dc9-86307b23d74f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:43.691344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21248c3b-719b-4c3d-8dc9-86307b23d74f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:43.712969 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1132/2307] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 20:54:43.717223 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebad8b0-9e2e-45a1-8432-5b556ac9c545 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:43.728772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebad8b0-9e2e-45a1-8432-5b556ac9c545 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:43.734609 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aafec2-dbab-4a12-bba1-c6ca2a39cabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:43.735299 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aafec2-dbab-4a12-bba1-c6ca2a39cabb admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:43.735595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aafec2-dbab-4a12-bba1-c6ca2a39cabb admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:43.735893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aafec2-dbab-4a12-bba1-c6ca2a39cabb admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:43.736870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aafec2-dbab-4a12-bba1-c6ca2a39cabb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:43.739001 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aafec2-dbab-4a12-bba1-c6ca2a39cabb admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:43.776289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1176/2308] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:43 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:54:43.780273 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21248c3b-719b-4c3d-8dc9-86307b23d74f admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:43.793058 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21248c3b-719b-4c3d-8dc9-86307b23d74f admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:43.857612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0e17e-1403-4fb8-af5d-e7a9787d546f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:43.858271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0e17e-1403-4fb8-af5d-e7a9787d546f admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:43.858648 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0e17e-1403-4fb8-af5d-e7a9787d546f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:43.858932 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0e17e-1403-4fb8-af5d-e7a9787d546f admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:43.859644 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0e17e-1403-4fb8-af5d-e7a9787d546f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:43.862096 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0e17e-1403-4fb8-af5d-e7a9787d546f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:43.887706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1133/2309] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:43 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:54:43.892582 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aafec2-dbab-4a12-bba1-c6ca2a39cabb admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:43.908734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aafec2-dbab-4a12-bba1-c6ca2a39cabb admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:43.914907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12346343-7459-48da-aad8-6df5b70b83e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:43.915854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12346343-7459-48da-aad8-6df5b70b83e9 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:43.916110 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12346343-7459-48da-aad8-6df5b70b83e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:43.916467 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12346343-7459-48da-aad8-6df5b70b83e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:43.917617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12346343-7459-48da-aad8-6df5b70b83e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:43.919241 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12346343-7459-48da-aad8-6df5b70b83e9 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:43.930894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1177/2310] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:43.934900 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0e17e-1403-4fb8-af5d-e7a9787d546f admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:43.946255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0e17e-1403-4fb8-af5d-e7a9787d546f admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:43.954588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f12b46-d171-4fd9-926f-dc4cd573db86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:43.955305 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f12b46-d171-4fd9-926f-dc4cd573db86 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:43.955596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f12b46-d171-4fd9-926f-dc4cd573db86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:43.955905 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f12b46-d171-4fd9-926f-dc4cd573db86 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:43.956664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f12b46-d171-4fd9-926f-dc4cd573db86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:43.958328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f12b46-d171-4fd9-926f-dc4cd573db86 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:43.973707 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1134/2311] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:54:43.974876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12346343-7459-48da-aad8-6df5b70b83e9 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:43.986651 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12346343-7459-48da-aad8-6df5b70b83e9 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:43.996397 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f134498-2c7d-4252-b34b-3076f87cc3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:43.997357 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f134498-2c7d-4252-b34b-3076f87cc3db admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:43.997600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f134498-2c7d-4252-b34b-3076f87cc3db admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:43.997881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f134498-2c7d-4252-b34b-3076f87cc3db admin admin] PATH_INFO: `/v3/projects/aa8a9fa0de9047759d68666f106b5763/users/f84b3dc69f0a4153a45cedc8c38620bc/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:44.001526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:44.021351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f134498-2c7d-4252-b34b-3076f87cc3db admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8a9fa0de9047759d68666f106b5763, user_id=f84b3dc69f0a4153a45cedc8c38620bc, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:44.024808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f134498-2c7d-4252-b34b-3076f87cc3db admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:44.040264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1178/2312] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:43 2022] PUT /identity/v3/projects/aa8a9fa0de9047759d68666f106b5763/users/f84b3dc69f0a4153a45cedc8c38620bc/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:44.056401 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630bac34-d74b-471c-a0c9-e7adf8cc2df0 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:44.056928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630bac34-d74b-471c-a0c9-e7adf8cc2df0 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:44.057468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630bac34-d74b-471c-a0c9-e7adf8cc2df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:44.096231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-630bac34-d74b-471c-a0c9-e7adf8cc2df0 None None] MFA Rules not processed for user `f84b3dc69f0a4153a45cedc8c38620bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:44.097531 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-630bac34-d74b-471c-a0c9-e7adf8cc2df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:44.200926 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1135/2313] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:44.213876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f134498-2c7d-4252-b34b-3076f87cc3db admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:44.216565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-5f134498-2c7d-4252-b34b-3076f87cc3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:44.367770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f134498-2c7d-4252-b34b-3076f87cc3db admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:44.388481 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18577c5-a71c-4211-9835-89387c9f3f52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:44.391296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18577c5-a71c-4211-9835-89387c9f3f52 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:44.393398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18577c5-a71c-4211-9835-89387c9f3f52 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:44.395004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18577c5-a71c-4211-9835-89387c9f3f52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:44.433668 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18577c5-a71c-4211-9835-89387c9f3f52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:44.437305 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18577c5-a71c-4211-9835-89387c9f3f52 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:44.462137 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1179/2314] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:54:44.490119 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630bac34-d74b-471c-a0c9-e7adf8cc2df0 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:44.540189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:44.587496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630bac34-d74b-471c-a0c9-e7adf8cc2df0 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:44.592753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983e9e9-561b-4faf-8d19-19760bd705db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:44.594062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983e9e9-561b-4faf-8d19-19760bd705db admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:44.594062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983e9e9-561b-4faf-8d19-19760bd705db admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:44.594323 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983e9e9-561b-4faf-8d19-19760bd705db admin admin] PATH_INFO: `/v3/users/f84b3dc69f0a4153a45cedc8c38620bc` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:44.596211 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983e9e9-561b-4faf-8d19-19760bd705db admin admin] RBAC: Authorizing `identity:delete_user(user_id=f84b3dc69f0a4153a45cedc8c38620bc)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:44.599152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983e9e9-561b-4faf-8d19-19760bd705db admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:44.669323 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-1983e9e9-561b-4faf-8d19-19760bd705db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f84b3dc69f0a4153a45cedc8c38620bc'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:44.686298 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-1983e9e9-561b-4faf-8d19-19760bd705db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f84b3dc69f0a4153a45cedc8c38620bc'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:44.686920 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-1983e9e9-561b-4faf-8d19-19760bd705db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f84b3dc69f0a4153a45cedc8c38620bc'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:44.697196 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-1983e9e9-561b-4faf-8d19-19760bd705db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f84b3dc69f0a4153a45cedc8c38620bc'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:44.698684 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-1983e9e9-561b-4faf-8d19-19760bd705db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f84b3dc69f0a4153a45cedc8c38620bc'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:44.724303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1136/2315] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:44 2022] DELETE /identity/v3/users/f84b3dc69f0a4153a45cedc8c38620bc => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:44.730804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff3e27-7c0f-4d62-83a3-81686695e1a9 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:44.731358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff3e27-7c0f-4d62-83a3-81686695e1a9 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:44.731449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff3e27-7c0f-4d62-83a3-81686695e1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:44.759322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-1cff3e27-7c0f-4d62-83a3-81686695e1a9 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:44.761350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1cff3e27-7c0f-4d62-83a3-81686695e1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:44.844139 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1180/2316] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:44.855850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983e9e9-561b-4faf-8d19-19760bd705db admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:44.858131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1983e9e9-561b-4faf-8d19-19760bd705db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:44.925659 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983e9e9-561b-4faf-8d19-19760bd705db admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:44.929806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca35e3-14e4-432d-8e87-9121bd511416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:44.930474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca35e3-14e4-432d-8e87-9121bd511416 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:44.930760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca35e3-14e4-432d-8e87-9121bd511416 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:44.931043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca35e3-14e4-432d-8e87-9121bd511416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:44.940837 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca35e3-14e4-432d-8e87-9121bd511416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:44.942289 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca35e3-14e4-432d-8e87-9121bd511416 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:44.952967 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1137/2317] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:45.547075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff3e27-7c0f-4d62-83a3-81686695e1a9 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:45.548868 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1cff3e27-7c0f-4d62-83a3-81686695e1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:45.567657 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff3e27-7c0f-4d62-83a3-81686695e1a9 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:45.577978 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c75ed-a8c6-4f68-a86f-be69f36783f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:45.579118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c75ed-a8c6-4f68-a86f-be69f36783f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:45.579454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c75ed-a8c6-4f68-a86f-be69f36783f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:45.579821 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c75ed-a8c6-4f68-a86f-be69f36783f9 admin admin] PATH_INFO: `/v3/projects/aa8a9fa0de9047759d68666f106b5763` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:45.582147 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c75ed-a8c6-4f68-a86f-be69f36783f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa8a9fa0de9047759d68666f106b5763)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:45.584356 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c75ed-a8c6-4f68-a86f-be69f36783f9 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:45.675780 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1d2c75ed-a8c6-4f68-a86f-be69f36783f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa8a9fa0de9047759d68666f106b5763', 'request_id': 'req-1d2c75ed-a8c6-4f68-a86f-be69f36783f9'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:45.683848 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1d2c75ed-a8c6-4f68-a86f-be69f36783f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa8a9fa0de9047759d68666f106b5763', 'request_id': 'req-1d2c75ed-a8c6-4f68-a86f-be69f36783f9'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:45.703825 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1d2c75ed-a8c6-4f68-a86f-be69f36783f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa8a9fa0de9047759d68666f106b5763', 'request_id': 'req-1d2c75ed-a8c6-4f68-a86f-be69f36783f9'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:45.731157 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1d2c75ed-a8c6-4f68-a86f-be69f36783f9 admin admin] The token cache is being invalidate because project aa8a9fa0de9047759d68666f106b5763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:45.732179 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1d2c75ed-a8c6-4f68-a86f-be69f36783f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:45.732378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1d2c75ed-a8c6-4f68-a86f-be69f36783f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:45.739625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1181/2318] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:45 2022] DELETE /identity/v3/projects/aa8a9fa0de9047759d68666f106b5763 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:45.741211 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4783800-5512-4361-98c6-db403c4e9445 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:45.741695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4783800-5512-4361-98c6-db403c4e9445 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:45.741806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4783800-5512-4361-98c6-db403c4e9445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:45.776539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-a4783800-5512-4361-98c6-db403c4e9445 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:45.779009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a4783800-5512-4361-98c6-db403c4e9445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:45.872688 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:45.918457 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1138/2319] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:54:45.927809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f19417-cd66-4b59-b966-e80b3c42b7a2 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:45.928505 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f19417-cd66-4b59-b966-e80b3c42b7a2 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:45.928901 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f19417-cd66-4b59-b966-e80b3c42b7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:45.950191 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-14f19417-cd66-4b59-b966-e80b3c42b7a2 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:45.951616 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-14f19417-cd66-4b59-b966-e80b3c42b7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:45.971897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1182/2320] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:45.973020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4783800-5512-4361-98c6-db403c4e9445 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:45.986598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4783800-5512-4361-98c6-db403c4e9445 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:45.992769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d10f30-f36f-4218-bc95-62377f149ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:45.993831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d10f30-f36f-4218-bc95-62377f149ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:45.994359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d10f30-f36f-4218-bc95-62377f149ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:45.994861 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d10f30-f36f-4218-bc95-62377f149ef4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:45.995930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d10f30-f36f-4218-bc95-62377f149ef4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:54:45.996617 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d10f30-f36f-4218-bc95-62377f149ef4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:45.998986 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d10f30-f36f-4218-bc95-62377f149ef4 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:46.016124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1139/2321] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:54:46.022781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f19417-cd66-4b59-b966-e80b3c42b7a2 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:46.028949 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:46.049354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f19417-cd66-4b59-b966-e80b3c42b7a2 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:46.055503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4506f-1ba1-443e-8c34-7f60f2cf27a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:46.056235 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4506f-1ba1-443e-8c34-7f60f2cf27a3 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:46.056522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4506f-1ba1-443e-8c34-7f60f2cf27a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:46.056802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4506f-1ba1-443e-8c34-7f60f2cf27a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:46.057556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4506f-1ba1-443e-8c34-7f60f2cf27a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:46.059683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4506f-1ba1-443e-8c34-7f60f2cf27a3 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:46.114848 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1183/2322] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:46 2022] POST /identity/v3/projects => generated 375 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:54:46.119213 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d10f30-f36f-4218-bc95-62377f149ef4 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:46.134277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d10f30-f36f-4218-bc95-62377f149ef4 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:46.220069 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66263161-8cf2-445c-b487-2b76d323685e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:46.220937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66263161-8cf2-445c-b487-2b76d323685e admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:46.221754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66263161-8cf2-445c-b487-2b76d323685e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:46.221754 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66263161-8cf2-445c-b487-2b76d323685e admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:46.222705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66263161-8cf2-445c-b487-2b76d323685e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:46.224985 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66263161-8cf2-445c-b487-2b76d323685e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:46.257763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1140/2323] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:46 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:46.262675 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4506f-1ba1-443e-8c34-7f60f2cf27a3 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:46.276768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4506f-1ba1-443e-8c34-7f60f2cf27a3 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:46.285187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3850f2-169b-4711-87da-7e0b4ea2de42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:46.286451 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3850f2-169b-4711-87da-7e0b4ea2de42 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:46.286957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3850f2-169b-4711-87da-7e0b4ea2de42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:46.288192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3850f2-169b-4711-87da-7e0b4ea2de42 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:46.289229 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3850f2-169b-4711-87da-7e0b4ea2de42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:46.291804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3850f2-169b-4711-87da-7e0b4ea2de42 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:46.314254 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1184/2324] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:46.318047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66263161-8cf2-445c-b487-2b76d323685e admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:46.330040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66263161-8cf2-445c-b487-2b76d323685e admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:46.335950 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dacdbd-2d37-452f-a654-29c5a8325e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:46.336698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dacdbd-2d37-452f-a654-29c5a8325e5c admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:46.336928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dacdbd-2d37-452f-a654-29c5a8325e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:46.337344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dacdbd-2d37-452f-a654-29c5a8325e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:46.338067 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dacdbd-2d37-452f-a654-29c5a8325e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:46.339597 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dacdbd-2d37-452f-a654-29c5a8325e5c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:46.351472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1141/2325] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:46.358297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3850f2-169b-4711-87da-7e0b4ea2de42 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:46.369689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3850f2-169b-4711-87da-7e0b4ea2de42 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:46.377973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16ad24-2b23-4267-8942-31105122fcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:46.378883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16ad24-2b23-4267-8942-31105122fcad admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:46.379163 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16ad24-2b23-4267-8942-31105122fcad admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:46.379442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16ad24-2b23-4267-8942-31105122fcad admin admin] PATH_INFO: `/v3/projects/a0f56ad01c544c958f8e65367ff5c575/users/5c6e45e0cb81448baaae7d4f2ccd5d28/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:46.421245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16ad24-2b23-4267-8942-31105122fcad admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f56ad01c544c958f8e65367ff5c575, user_id=5c6e45e0cb81448baaae7d4f2ccd5d28, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:46.424450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16ad24-2b23-4267-8942-31105122fcad admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:46.465972 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1185/2326] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:46 2022] PUT /identity/v3/projects/a0f56ad01c544c958f8e65367ff5c575/users/5c6e45e0cb81448baaae7d4f2ccd5d28/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:46.478814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0654cf-1701-4a50-aaba-164538a6a6b3 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:46.479652 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0654cf-1701-4a50-aaba-164538a6a6b3 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:46.480579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0654cf-1701-4a50-aaba-164538a6a6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:46.542028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-ea0654cf-1701-4a50-aaba-164538a6a6b3 None None] MFA Rules not processed for user `5c6e45e0cb81448baaae7d4f2ccd5d28`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:46.543846 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ea0654cf-1701-4a50-aaba-164538a6a6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:46.656091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1142/2327] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:54:46.662327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16ad24-2b23-4267-8942-31105122fcad admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:46.667981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9f16ad24-2b23-4267-8942-31105122fcad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:46.668180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0654cf-1701-4a50-aaba-164538a6a6b3 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:46.669809 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ea0654cf-1701-4a50-aaba-164538a6a6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:46.742308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:46.747841 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0654cf-1701-4a50-aaba-164538a6a6b3 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:46.754928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414e591-e310-42b6-8f8d-9de359cd7337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:46.755203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16ad24-2b23-4267-8942-31105122fcad admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:46.756010 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414e591-e310-42b6-8f8d-9de359cd7337 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:46.756504 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414e591-e310-42b6-8f8d-9de359cd7337 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:46.756974 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414e591-e310-42b6-8f8d-9de359cd7337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:46.763283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96731812-75c7-4177-803a-0f4781dbc09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:46.764885 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96731812-75c7-4177-803a-0f4781dbc09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:46.765354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96731812-75c7-4177-803a-0f4781dbc09f admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:46.765816 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96731812-75c7-4177-803a-0f4781dbc09f admin admin] PATH_INFO: `/v3/users/bc6b0bfbda8944369d1a83b52b400f64` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:46.767897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96731812-75c7-4177-803a-0f4781dbc09f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc6b0bfbda8944369d1a83b52b400f64)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:46.771337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96731812-75c7-4177-803a-0f4781dbc09f admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:46.775340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414e591-e310-42b6-8f8d-9de359cd7337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:46.777757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414e591-e310-42b6-8f8d-9de359cd7337 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:46.797132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1143/2328] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:54:46.846234 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-96731812-75c7-4177-803a-0f4781dbc09f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc6b0bfbda8944369d1a83b52b400f64'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:46.847734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-96731812-75c7-4177-803a-0f4781dbc09f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc6b0bfbda8944369d1a83b52b400f64'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:46.862219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-96731812-75c7-4177-803a-0f4781dbc09f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc6b0bfbda8944369d1a83b52b400f64'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:46.870941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-96731812-75c7-4177-803a-0f4781dbc09f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc6b0bfbda8944369d1a83b52b400f64'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:46.878323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-96731812-75c7-4177-803a-0f4781dbc09f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc6b0bfbda8944369d1a83b52b400f64'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:46.885173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1186/2329] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:46 2022] DELETE /identity/v3/users/bc6b0bfbda8944369d1a83b52b400f64 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:54:47.124133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414e591-e310-42b6-8f8d-9de359cd7337 service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:47.144896 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414e591-e310-42b6-8f8d-9de359cd7337 service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:47.305755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f1e1e-449a-472b-bf9f-9d42a3ac9184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:47.307004 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f1e1e-449a-472b-bf9f-9d42a3ac9184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:47.307520 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f1e1e-449a-472b-bf9f-9d42a3ac9184 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:47.308013 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f1e1e-449a-472b-bf9f-9d42a3ac9184 admin admin] PATH_INFO: `/v3/projects/4fe52006c5034bbf8de92b8aae171d3c` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:47.309629 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f1e1e-449a-472b-bf9f-9d42a3ac9184 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fe52006c5034bbf8de92b8aae171d3c)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:47.311681 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f1e1e-449a-472b-bf9f-9d42a3ac9184 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:47.369051 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-240f1e1e-449a-472b-bf9f-9d42a3ac9184 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fe52006c5034bbf8de92b8aae171d3c', 'request_id': 'req-240f1e1e-449a-472b-bf9f-9d42a3ac9184'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:47.382105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-240f1e1e-449a-472b-bf9f-9d42a3ac9184 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fe52006c5034bbf8de92b8aae171d3c', 'request_id': 'req-240f1e1e-449a-472b-bf9f-9d42a3ac9184'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:47.389023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-240f1e1e-449a-472b-bf9f-9d42a3ac9184 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fe52006c5034bbf8de92b8aae171d3c', 'request_id': 'req-240f1e1e-449a-472b-bf9f-9d42a3ac9184'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:47.406059 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-240f1e1e-449a-472b-bf9f-9d42a3ac9184 admin admin] The token cache is being invalidate because project 4fe52006c5034bbf8de92b8aae171d3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:47.406363 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-240f1e1e-449a-472b-bf9f-9d42a3ac9184 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:47.407032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-240f1e1e-449a-472b-bf9f-9d42a3ac9184 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:47.409042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1144/2330] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:47 2022] DELETE /identity/v3/projects/4fe52006c5034bbf8de92b8aae171d3c => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:47.415383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d681be-b379-4ea8-84b2-a19f1f00b2e6 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:47.415780 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d681be-b379-4ea8-84b2-a19f1f00b2e6 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:47.416164 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d681be-b379-4ea8-84b2-a19f1f00b2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:47.436328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-31d681be-b379-4ea8-84b2-a19f1f00b2e6 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:47.437557 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-31d681be-b379-4ea8-84b2-a19f1f00b2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:47.457989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:47.508323 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1187/2331] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:54:47.516235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809ba03-82b7-4dcf-9765-5bb20085af46 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:47.516529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809ba03-82b7-4dcf-9765-5bb20085af46 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:47.516815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809ba03-82b7-4dcf-9765-5bb20085af46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:47.539279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-4809ba03-82b7-4dcf-9765-5bb20085af46 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:47.540495 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-4809ba03-82b7-4dcf-9765-5bb20085af46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:47.552537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1145/2332] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:47.558020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d681be-b379-4ea8-84b2-a19f1f00b2e6 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:47.571411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d681be-b379-4ea8-84b2-a19f1f00b2e6 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:47.577665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c279c-5d9c-4ec8-bc43-d05334c0819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:47.578470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c279c-5d9c-4ec8-bc43-d05334c0819a admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:47.578788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c279c-5d9c-4ec8-bc43-d05334c0819a admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:47.579502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c279c-5d9c-4ec8-bc43-d05334c0819a admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:47.579823 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c279c-5d9c-4ec8-bc43-d05334c0819a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:54:47.580132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c279c-5d9c-4ec8-bc43-d05334c0819a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:47.581733 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c279c-5d9c-4ec8-bc43-d05334c0819a admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:47.595893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1188/2333] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:54:47.601249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809ba03-82b7-4dcf-9765-5bb20085af46 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:47.610838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809ba03-82b7-4dcf-9765-5bb20085af46 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:47.616527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85608d-f598-4600-907c-445de0dbc223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:47.617541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85608d-f598-4600-907c-445de0dbc223 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:47.617890 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85608d-f598-4600-907c-445de0dbc223 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:47.618176 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85608d-f598-4600-907c-445de0dbc223 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:47.618919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85608d-f598-4600-907c-445de0dbc223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:47.621138 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85608d-f598-4600-907c-445de0dbc223 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:47.653510 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1146/2334] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:47 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:47.656282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c279c-5d9c-4ec8-bc43-d05334c0819a admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:47.666570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c279c-5d9c-4ec8-bc43-d05334c0819a admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:47.728310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e3eb9-9585-4dc6-9281-8e9911abb3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:47.728997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e3eb9-9585-4dc6-9281-8e9911abb3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:47.729278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e3eb9-9585-4dc6-9281-8e9911abb3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:47.729555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e3eb9-9585-4dc6-9281-8e9911abb3a5 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:47.730709 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e3eb9-9585-4dc6-9281-8e9911abb3a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:47.732336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e3eb9-9585-4dc6-9281-8e9911abb3a5 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:47.754128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1189/2335] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:47 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:54:47.760122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85608d-f598-4600-907c-445de0dbc223 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:47.770777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85608d-f598-4600-907c-445de0dbc223 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:47.775692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e4518-5c92-4d7e-a059-c83cb77c30a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:47.776385 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e4518-5c92-4d7e-a059-c83cb77c30a7 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:47.776669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e4518-5c92-4d7e-a059-c83cb77c30a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:47.776948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e4518-5c92-4d7e-a059-c83cb77c30a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:47.777672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e4518-5c92-4d7e-a059-c83cb77c30a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:47.779328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e4518-5c92-4d7e-a059-c83cb77c30a7 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:47.791809 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1147/2336] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:47.796134 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e3eb9-9585-4dc6-9281-8e9911abb3a5 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:47.807639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e3eb9-9585-4dc6-9281-8e9911abb3a5 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:47.813085 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91ccbd-361c-4d5a-9275-3d759d488559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:47.813815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91ccbd-361c-4d5a-9275-3d759d488559 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:47.814111 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91ccbd-361c-4d5a-9275-3d759d488559 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:47.814406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91ccbd-361c-4d5a-9275-3d759d488559 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:47.815173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91ccbd-361c-4d5a-9275-3d759d488559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:47.816901 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91ccbd-361c-4d5a-9275-3d759d488559 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:47.830862 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1190/2337] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:54:47.835365 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e4518-5c92-4d7e-a059-c83cb77c30a7 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:47.845791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e4518-5c92-4d7e-a059-c83cb77c30a7 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:47.851100 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c0f30-4f74-4773-b3e6-c371e4d991fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:47.851952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c0f30-4f74-4773-b3e6-c371e4d991fb admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:47.852243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c0f30-4f74-4773-b3e6-c371e4d991fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:47.852608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c0f30-4f74-4773-b3e6-c371e4d991fb admin admin] PATH_INFO: `/v3/projects/c51d3542adae4062a487ab85d673140d/users/ab2bc1eb1eb845ec82b79adee04a51e7/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:47.868819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c0f30-4f74-4773-b3e6-c371e4d991fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c51d3542adae4062a487ab85d673140d, user_id=ab2bc1eb1eb845ec82b79adee04a51e7, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:47.871564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c0f30-4f74-4773-b3e6-c371e4d991fb admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:47.885158 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1148/2338] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:54:47 2022] PUT /identity/v3/projects/c51d3542adae4062a487ab85d673140d/users/ab2bc1eb1eb845ec82b79adee04a51e7/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:54:47.891960 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4aaf9-13b2-421f-a691-dca25d22a8f7 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:47.892404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4aaf9-13b2-421f-a691-dca25d22a8f7 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:47.892829 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4aaf9-13b2-421f-a691-dca25d22a8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:47.917979 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-85c4aaf9-13b2-421f-a691-dca25d22a8f7 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:47.919411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-85c4aaf9-13b2-421f-a691-dca25d22a8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:47.994322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1191/2339] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:48.003998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c0f30-4f74-4773-b3e6-c371e4d991fb admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:48.005738 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-967c0f30-4f74-4773-b3e6-c371e4d991fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:48.033047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:48.076518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c0f30-4f74-4773-b3e6-c371e4d991fb admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:48.080280 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad514e7-0e88-42c3-bf27-990c25e3990f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:48.080934 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad514e7-0e88-42c3-bf27-990c25e3990f service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:48.081288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad514e7-0e88-42c3-bf27-990c25e3990f service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:48.081495 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad514e7-0e88-42c3-bf27-990c25e3990f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:48.091064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad514e7-0e88-42c3-bf27-990c25e3990f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:48.092546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad514e7-0e88-42c3-bf27-990c25e3990f service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:48.102955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1149/2340] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:49.401075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155a8ac-07d6-4069-9b96-02008cfcf5d7 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:49.401607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155a8ac-07d6-4069-9b96-02008cfcf5d7 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:49.402098 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155a8ac-07d6-4069-9b96-02008cfcf5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:49.439781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-9155a8ac-07d6-4069-9b96-02008cfcf5d7 None None] MFA Rules not processed for user `5c6e45e0cb81448baaae7d4f2ccd5d28`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:49.441709 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9155a8ac-07d6-4069-9b96-02008cfcf5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:49.483227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:49.557579 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1192/2341] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:54.077169 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1610e-cb03-47d5-bf97-1a0a2a01fcc3 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:54.077705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1610e-cb03-47d5-bf97-1a0a2a01fcc3 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:54.077705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1610e-cb03-47d5-bf97-1a0a2a01fcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:54.113358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c5b1610e-cb03-47d5-bf97-1a0a2a01fcc3 None None] MFA Rules not processed for user `ab2bc1eb1eb845ec82b79adee04a51e7`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:54.115210 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c5b1610e-cb03-47d5-bf97-1a0a2a01fcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:54.228289 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1150/2342] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:54.237919 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155a8ac-07d6-4069-9b96-02008cfcf5d7 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:54.240267 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9155a8ac-07d6-4069-9b96-02008cfcf5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:54.387557 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155a8ac-07d6-4069-9b96-02008cfcf5d7 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:54.395519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38460fc2-f820-4ad3-8a2a-2d1d3b8deeeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:54.396239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38460fc2-f820-4ad3-8a2a-2d1d3b8deeeb service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:54.397210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38460fc2-f820-4ad3-8a2a-2d1d3b8deeeb service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:54.397507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38460fc2-f820-4ad3-8a2a-2d1d3b8deeeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:54.418728 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38460fc2-f820-4ad3-8a2a-2d1d3b8deeeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:54.420535 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38460fc2-f820-4ad3-8a2a-2d1d3b8deeeb service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:54.442799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1193/2343] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:54:56.244086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2d467-6b8d-412c-b972-9cb1239191c9 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:56.244643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2d467-6b8d-412c-b972-9cb1239191c9 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:56.244643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2d467-6b8d-412c-b972-9cb1239191c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:56.265253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-43d2d467-6b8d-412c-b972-9cb1239191c9 None None] MFA Rules not processed for user `ab2bc1eb1eb845ec82b79adee04a51e7`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:56.266760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-43d2d467-6b8d-412c-b972-9cb1239191c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:56.284493 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1151/2344] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:57.228321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea5223-af00-4a36-acae-59ec556aac45 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:57.228763 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea5223-af00-4a36-acae-59ec556aac45 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:57.228890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea5223-af00-4a36-acae-59ec556aac45 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:57.230882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1194/2345] 10.222.0.60 () {54 vars in 912 bytes} [Thu Jul 21 20:54:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:54:57.236914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37bf197-7e98-4fc8-9fd2-841abae2ff1a None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:57.237438 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37bf197-7e98-4fc8-9fd2-841abae2ff1a None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:57.237899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37bf197-7e98-4fc8-9fd2-841abae2ff1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:57.273325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-f37bf197-7e98-4fc8-9fd2-841abae2ff1a None None] MFA Rules not processed for user `33aff48933844401b86eb808937e35c4`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:57.275395 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f37bf197-7e98-4fc8-9fd2-841abae2ff1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:57.291231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1152/2346] 10.222.0.60 () {58 vars in 1009 bytes} [Thu Jul 21 20:54:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:57.303588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea5223-af00-4a36-acae-59ec556aac45 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:57.322174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea5223-af00-4a36-acae-59ec556aac45 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:57.327215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef72de6-0a89-4079-8d21-a33781d98237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:57.328228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef72de6-0a89-4079-8d21-a33781d98237 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:57.328861 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef72de6-0a89-4079-8d21-a33781d98237 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:57.329356 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef72de6-0a89-4079-8d21-a33781d98237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:57.367392 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef72de6-0a89-4079-8d21-a33781d98237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:57.368915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef72de6-0a89-4079-8d21-a33781d98237 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:57.397751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1195/2347] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:54:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:54:57.941668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37bf197-7e98-4fc8-9fd2-841abae2ff1a None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:57.944240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f37bf197-7e98-4fc8-9fd2-841abae2ff1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:57.964793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37bf197-7e98-4fc8-9fd2-841abae2ff1a None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:57.970076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86727f70-0c67-4d12-af49-394d02eabbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:57.971174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86727f70-0c67-4d12-af49-394d02eabbe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:57.971831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86727f70-0c67-4d12-af49-394d02eabbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:57.972101 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86727f70-0c67-4d12-af49-394d02eabbe1 admin admin] PATH_INFO: `/v3/users/a146c0d3471b4feab7a02d5e0a1e2bde` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:57.973454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86727f70-0c67-4d12-af49-394d02eabbe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a146c0d3471b4feab7a02d5e0a1e2bde)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:57.978330 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86727f70-0c67-4d12-af49-394d02eabbe1 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:58.076642 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-86727f70-0c67-4d12-af49-394d02eabbe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a146c0d3471b4feab7a02d5e0a1e2bde'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:58.100136 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-86727f70-0c67-4d12-af49-394d02eabbe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a146c0d3471b4feab7a02d5e0a1e2bde'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:58.101031 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-86727f70-0c67-4d12-af49-394d02eabbe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a146c0d3471b4feab7a02d5e0a1e2bde'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:58.111338 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-86727f70-0c67-4d12-af49-394d02eabbe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a146c0d3471b4feab7a02d5e0a1e2bde'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:58.113044 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-86727f70-0c67-4d12-af49-394d02eabbe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a146c0d3471b4feab7a02d5e0a1e2bde'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:58.138263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1153/2348] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:57 2022] DELETE /identity/v3/users/a146c0d3471b4feab7a02d5e0a1e2bde => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:58.430782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef72de6-0a89-4079-8d21-a33781d98237 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:58.468389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef72de6-0a89-4079-8d21-a33781d98237 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:58.607513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0943e-829f-45cb-afbf-cda71e32ee06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:58.609169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0943e-829f-45cb-afbf-cda71e32ee06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:58.610826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0943e-829f-45cb-afbf-cda71e32ee06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:58.611644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0943e-829f-45cb-afbf-cda71e32ee06 admin admin] PATH_INFO: `/v3/projects/91082d5e6cc94246b34b31b7518fb8d1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:58.613273 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0943e-829f-45cb-afbf-cda71e32ee06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91082d5e6cc94246b34b31b7518fb8d1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:58.616140 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0943e-829f-45cb-afbf-cda71e32ee06 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:58.695166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-43a0943e-829f-45cb-afbf-cda71e32ee06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91082d5e6cc94246b34b31b7518fb8d1', 'request_id': 'req-43a0943e-829f-45cb-afbf-cda71e32ee06'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:58.696617 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:58.703553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-43a0943e-829f-45cb-afbf-cda71e32ee06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91082d5e6cc94246b34b31b7518fb8d1', 'request_id': 'req-43a0943e-829f-45cb-afbf-cda71e32ee06'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:58.715893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-43a0943e-829f-45cb-afbf-cda71e32ee06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91082d5e6cc94246b34b31b7518fb8d1', 'request_id': 'req-43a0943e-829f-45cb-afbf-cda71e32ee06'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:58.737428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-43a0943e-829f-45cb-afbf-cda71e32ee06 admin admin] The token cache is being invalidate because project 91082d5e6cc94246b34b31b7518fb8d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:58.738264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-43a0943e-829f-45cb-afbf-cda71e32ee06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:58.739466 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-43a0943e-829f-45cb-afbf-cda71e32ee06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:58.743702 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1196/2349] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:58 2022] DELETE /identity/v3/projects/91082d5e6cc94246b34b31b7518fb8d1 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:58.750805 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86727f70-0c67-4d12-af49-394d02eabbe1 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:58.754629 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-86727f70-0c67-4d12-af49-394d02eabbe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:58.772523 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86727f70-0c67-4d12-af49-394d02eabbe1 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:58.789990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:54:58.869489 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03458827-bcae-48d0-b0f8-b64dc79229ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:58.870672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03458827-bcae-48d0-b0f8-b64dc79229ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:58.871068 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03458827-bcae-48d0-b0f8-b64dc79229ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:58.871662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03458827-bcae-48d0-b0f8-b64dc79229ed admin admin] PATH_INFO: `/v3/users/5ebfb2db400246048eb5107677aeee25` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:58.873637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03458827-bcae-48d0-b0f8-b64dc79229ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ebfb2db400246048eb5107677aeee25)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:58.877763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03458827-bcae-48d0-b0f8-b64dc79229ed admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:58.942177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-03458827-bcae-48d0-b0f8-b64dc79229ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ebfb2db400246048eb5107677aeee25'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:58.952627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-03458827-bcae-48d0-b0f8-b64dc79229ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ebfb2db400246048eb5107677aeee25'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:58.953475 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-03458827-bcae-48d0-b0f8-b64dc79229ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ebfb2db400246048eb5107677aeee25'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:58.963017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-03458827-bcae-48d0-b0f8-b64dc79229ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ebfb2db400246048eb5107677aeee25'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:58.964015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-03458827-bcae-48d0-b0f8-b64dc79229ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ebfb2db400246048eb5107677aeee25'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:58.985411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1154/2350] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:54:58 2022] DELETE /identity/v3/users/5ebfb2db400246048eb5107677aeee25 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:59.322221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0943e-829f-45cb-afbf-cda71e32ee06 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:59.325038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-43a0943e-829f-45cb-afbf-cda71e32ee06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:59.354252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0943e-829f-45cb-afbf-cda71e32ee06 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:59.485326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba371fc-78ab-4662-8b53-ed0f7a85d714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:59.486487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba371fc-78ab-4662-8b53-ed0f7a85d714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:59.486975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba371fc-78ab-4662-8b53-ed0f7a85d714 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:59.487314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba371fc-78ab-4662-8b53-ed0f7a85d714 admin admin] PATH_INFO: `/v3/projects/b03b888d646c47afa4da0a23fca675eb` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:59.488533 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba371fc-78ab-4662-8b53-ed0f7a85d714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b03b888d646c47afa4da0a23fca675eb)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:59.491285 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba371fc-78ab-4662-8b53-ed0f7a85d714 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:59.569390 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8ba371fc-78ab-4662-8b53-ed0f7a85d714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b03b888d646c47afa4da0a23fca675eb', 'request_id': 'req-8ba371fc-78ab-4662-8b53-ed0f7a85d714'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:59.577848 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8ba371fc-78ab-4662-8b53-ed0f7a85d714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b03b888d646c47afa4da0a23fca675eb', 'request_id': 'req-8ba371fc-78ab-4662-8b53-ed0f7a85d714'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:59.587683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8ba371fc-78ab-4662-8b53-ed0f7a85d714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b03b888d646c47afa4da0a23fca675eb', 'request_id': 'req-8ba371fc-78ab-4662-8b53-ed0f7a85d714'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:59.612556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8ba371fc-78ab-4662-8b53-ed0f7a85d714 admin admin] The token cache is being invalidate because project b03b888d646c47afa4da0a23fca675eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:54:59.613158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8ba371fc-78ab-4662-8b53-ed0f7a85d714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:59.614197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8ba371fc-78ab-4662-8b53-ed0f7a85d714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:54:59.617183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1197/2351] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:54:59 2022] DELETE /identity/v3/projects/b03b888d646c47afa4da0a23fca675eb => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:54:59.622983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3be64b-cb7a-4374-9c99-1b95bb8f8008 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:59.623645 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3be64b-cb7a-4374-9c99-1b95bb8f8008 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:59.624117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3be64b-cb7a-4374-9c99-1b95bb8f8008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:59.645358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-be3be64b-cb7a-4374-9c99-1b95bb8f8008 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:59.647400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-be3be64b-cb7a-4374-9c99-1b95bb8f8008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:59.785503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1155/2352] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:59.799363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce92d3-1048-411d-83ad-6c0ecf25e9a3 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:59.799866 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce92d3-1048-411d-83ad-6c0ecf25e9a3 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:59.800564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce92d3-1048-411d-83ad-6c0ecf25e9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:59.828136 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-dbce92d3-1048-411d-83ad-6c0ecf25e9a3 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:54:59.829595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-dbce92d3-1048-411d-83ad-6c0ecf25e9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:54:59.840307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1198/2353] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:54:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:54:59.846014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3be64b-cb7a-4374-9c99-1b95bb8f8008 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:59.858410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3be64b-cb7a-4374-9c99-1b95bb8f8008 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:59.865005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b62894-f14a-43a9-b59e-11f525250543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:59.865919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b62894-f14a-43a9-b59e-11f525250543 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:59.866452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b62894-f14a-43a9-b59e-11f525250543 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:59.866929 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b62894-f14a-43a9-b59e-11f525250543 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:59.867904 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b62894-f14a-43a9-b59e-11f525250543 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:54:59.868222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b62894-f14a-43a9-b59e-11f525250543 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:59.869860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b62894-f14a-43a9-b59e-11f525250543 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:59.891978 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1156/2354] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:54:59.895468 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce92d3-1048-411d-83ad-6c0ecf25e9a3 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:59.908507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce92d3-1048-411d-83ad-6c0ecf25e9a3 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:54:59.919883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95be573-33db-4de8-94f5-c42d51a79f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:54:59.920928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95be573-33db-4de8-94f5-c42d51a79f24 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:54:59.921373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95be573-33db-4de8-94f5-c42d51a79f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:54:59.921658 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95be573-33db-4de8-94f5-c42d51a79f24 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:54:59.922728 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95be573-33db-4de8-94f5-c42d51a79f24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:54:59.925206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95be573-33db-4de8-94f5-c42d51a79f24 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:54:59.965957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1199/2355] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:54:59 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:54:59.972857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b62894-f14a-43a9-b59e-11f525250543 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:54:59.988561 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b62894-f14a-43a9-b59e-11f525250543 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:00.020398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:00.133265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294ccd7-a7db-4bba-8950-c03002675b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:00.134540 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294ccd7-a7db-4bba-8950-c03002675b22 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:00.135217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294ccd7-a7db-4bba-8950-c03002675b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:00.135828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294ccd7-a7db-4bba-8950-c03002675b22 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:00.137717 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294ccd7-a7db-4bba-8950-c03002675b22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:00.141233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294ccd7-a7db-4bba-8950-c03002675b22 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:00.201572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1157/2356] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:54:59 2022] POST /identity/v3/users => generated 288 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:55:00.203268 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95be573-33db-4de8-94f5-c42d51a79f24 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:00.221755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95be573-33db-4de8-94f5-c42d51a79f24 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:00.234390 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43aec9-7a8e-48f5-839d-93351e518ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:00.235304 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43aec9-7a8e-48f5-839d-93351e518ebe admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:00.235590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43aec9-7a8e-48f5-839d-93351e518ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:00.235923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43aec9-7a8e-48f5-839d-93351e518ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:00.236928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43aec9-7a8e-48f5-839d-93351e518ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:00.239111 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43aec9-7a8e-48f5-839d-93351e518ebe admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:00.278685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1200/2357] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:55:00.284956 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294ccd7-a7db-4bba-8950-c03002675b22 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:00.321174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294ccd7-a7db-4bba-8950-c03002675b22 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:00.339071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227d91e-7b2a-440e-870c-63208374f28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:00.339905 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227d91e-7b2a-440e-870c-63208374f28a admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:00.340336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227d91e-7b2a-440e-870c-63208374f28a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:00.340665 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227d91e-7b2a-440e-870c-63208374f28a admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:00.342035 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227d91e-7b2a-440e-870c-63208374f28a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:00.354635 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227d91e-7b2a-440e-870c-63208374f28a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:00.394606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1158/2358] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:55:00.394793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43aec9-7a8e-48f5-839d-93351e518ebe admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:00.417441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43aec9-7a8e-48f5-839d-93351e518ebe admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:00.435887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a047c-e463-4ac2-b50f-0b7894a06f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:00.437143 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a047c-e463-4ac2-b50f-0b7894a06f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:00.437509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a047c-e463-4ac2-b50f-0b7894a06f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:00.437894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a047c-e463-4ac2-b50f-0b7894a06f4b admin admin] PATH_INFO: `/v3/projects/e42314cf48944558a247b652c87a9b5d/users/7187dab7be7f437ebbd751d200b5f0c9/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:00.462825 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a047c-e463-4ac2-b50f-0b7894a06f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e42314cf48944558a247b652c87a9b5d, user_id=7187dab7be7f437ebbd751d200b5f0c9, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:00.466516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a047c-e463-4ac2-b50f-0b7894a06f4b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:00.484903 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1201/2359] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:55:00 2022] PUT /identity/v3/projects/e42314cf48944558a247b652c87a9b5d/users/7187dab7be7f437ebbd751d200b5f0c9/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:55:00.505231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74f193-7e60-42af-8c2a-c207bc9c3453 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:00.505502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74f193-7e60-42af-8c2a-c207bc9c3453 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:00.505814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74f193-7e60-42af-8c2a-c207bc9c3453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:00.596389 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-eb74f193-7e60-42af-8c2a-c207bc9c3453 None None] MFA Rules not processed for user `7187dab7be7f437ebbd751d200b5f0c9`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:00.597731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-eb74f193-7e60-42af-8c2a-c207bc9c3453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:00.713333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1159/2360] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:55:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:55:00.728654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a047c-e463-4ac2-b50f-0b7894a06f4b admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:00.731685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-359a047c-e463-4ac2-b50f-0b7894a06f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:00.741521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:00.823969 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a047c-e463-4ac2-b50f-0b7894a06f4b admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:00.828657 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b84cce-86c7-4ecf-8d3c-cd1b9e7c05f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:00.829337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b84cce-86c7-4ecf-8d3c-cd1b9e7c05f0 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:00.829626 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b84cce-86c7-4ecf-8d3c-cd1b9e7c05f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:00.830201 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b84cce-86c7-4ecf-8d3c-cd1b9e7c05f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:00.846319 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b84cce-86c7-4ecf-8d3c-cd1b9e7c05f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:00.848117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b84cce-86c7-4ecf-8d3c-cd1b9e7c05f0 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:00.859975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1202/2361] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:55:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:55:10.710918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:10.854692 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=24 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:33.831016 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74f193-7e60-42af-8c2a-c207bc9c3453 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:33.834879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-eb74f193-7e60-42af-8c2a-c207bc9c3453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:33.925657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74f193-7e60-42af-8c2a-c207bc9c3453 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:33.932071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872ddbd-665e-4a7d-bdf7-31224de62461 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:33.932749 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872ddbd-665e-4a7d-bdf7-31224de62461 service placement] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:33.933077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872ddbd-665e-4a7d-bdf7-31224de62461 service placement] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:33.933394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872ddbd-665e-4a7d-bdf7-31224de62461 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:33.935968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8872ddbd-665e-4a7d-bdf7-31224de62461 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:33.949810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872ddbd-665e-4a7d-bdf7-31224de62461 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:33.951415 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872ddbd-665e-4a7d-bdf7-31224de62461 service placement] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:33.966936 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1160/2362] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:55:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:55:38.468107 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b84cce-86c7-4ecf-8d3c-cd1b9e7c05f0 service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:38.469569 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-63b84cce-86c7-4ecf-8d3c-cd1b9e7c05f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:38.548186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b84cce-86c7-4ecf-8d3c-cd1b9e7c05f0 service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:38.553215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c62bdf-ac72-4419-b985-cbda6b384529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:38.554278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c62bdf-ac72-4419-b985-cbda6b384529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:38.554585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c62bdf-ac72-4419-b985-cbda6b384529 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:38.554835 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c62bdf-ac72-4419-b985-cbda6b384529 admin admin] PATH_INFO: `/v3/users/fe76dcaa953f40ab890978e5c30bec65` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:38.556194 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c62bdf-ac72-4419-b985-cbda6b384529 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe76dcaa953f40ab890978e5c30bec65)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:38.558173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c62bdf-ac72-4419-b985-cbda6b384529 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:38.612047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-13c62bdf-ac72-4419-b985-cbda6b384529 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe76dcaa953f40ab890978e5c30bec65'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:38.612707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-13c62bdf-ac72-4419-b985-cbda6b384529 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe76dcaa953f40ab890978e5c30bec65'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:38.625658 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-13c62bdf-ac72-4419-b985-cbda6b384529 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe76dcaa953f40ab890978e5c30bec65'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:38.634591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-13c62bdf-ac72-4419-b985-cbda6b384529 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe76dcaa953f40ab890978e5c30bec65'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:38.642687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-13c62bdf-ac72-4419-b985-cbda6b384529 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe76dcaa953f40ab890978e5c30bec65'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:38.650244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1203/2363] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:55:38 2022] DELETE /identity/v3/users/fe76dcaa953f40ab890978e5c30bec65 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:55:38.655000 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca586462-f9bd-4dab-9dbe-e75962bc1f0d None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:38.655671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca586462-f9bd-4dab-9dbe-e75962bc1f0d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:38.655671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca586462-f9bd-4dab-9dbe-e75962bc1f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:38.674904 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-ca586462-f9bd-4dab-9dbe-e75962bc1f0d None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:38.676176 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ca586462-f9bd-4dab-9dbe-e75962bc1f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:38.745924 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1161/2364] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:55:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:55:38.757805 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c62bdf-ac72-4419-b985-cbda6b384529 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:38.759413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-13c62bdf-ac72-4419-b985-cbda6b384529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:38.839502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c62bdf-ac72-4419-b985-cbda6b384529 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:38.843399 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f887bb-a48c-4146-a2e9-84a47f389606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:38.844101 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f887bb-a48c-4146-a2e9-84a47f389606 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:38.844387 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f887bb-a48c-4146-a2e9-84a47f389606 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:38.844664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f887bb-a48c-4146-a2e9-84a47f389606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:38.856259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f887bb-a48c-4146-a2e9-84a47f389606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:38.857700 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f887bb-a48c-4146-a2e9-84a47f389606 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:38.869019 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1204/2365] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:55:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:55:39.220390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca586462-f9bd-4dab-9dbe-e75962bc1f0d None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:39.221865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ca586462-f9bd-4dab-9dbe-e75962bc1f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:39.236369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca586462-f9bd-4dab-9dbe-e75962bc1f0d None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:39.241872 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee99ec0-ff02-4dce-829b-01aac6bf4cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:39.242766 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee99ec0-ff02-4dce-829b-01aac6bf4cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:39.243056 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee99ec0-ff02-4dce-829b-01aac6bf4cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:39.243394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee99ec0-ff02-4dce-829b-01aac6bf4cc7 admin admin] PATH_INFO: `/v3/projects/546d785812474b61a9ad792dc26afcd8` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:39.244703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee99ec0-ff02-4dce-829b-01aac6bf4cc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=546d785812474b61a9ad792dc26afcd8)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:39.246637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee99ec0-ff02-4dce-829b-01aac6bf4cc7 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:39.301379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-4ee99ec0-ff02-4dce-829b-01aac6bf4cc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '546d785812474b61a9ad792dc26afcd8', 'request_id': 'req-4ee99ec0-ff02-4dce-829b-01aac6bf4cc7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:39.310013 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-4ee99ec0-ff02-4dce-829b-01aac6bf4cc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '546d785812474b61a9ad792dc26afcd8', 'request_id': 'req-4ee99ec0-ff02-4dce-829b-01aac6bf4cc7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:39.316259 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-4ee99ec0-ff02-4dce-829b-01aac6bf4cc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '546d785812474b61a9ad792dc26afcd8', 'request_id': 'req-4ee99ec0-ff02-4dce-829b-01aac6bf4cc7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:39.328512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-4ee99ec0-ff02-4dce-829b-01aac6bf4cc7 admin admin] The token cache is being invalidate because project 546d785812474b61a9ad792dc26afcd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:55:39.328827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-4ee99ec0-ff02-4dce-829b-01aac6bf4cc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:39.329511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-4ee99ec0-ff02-4dce-829b-01aac6bf4cc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:39.331128 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1162/2366] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:55:39 2022] DELETE /identity/v3/projects/546d785812474b61a9ad792dc26afcd8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:55:39.336890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbc13b-aaa8-4257-94ff-d74aa5789e0d None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:39.337176 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbc13b-aaa8-4257-94ff-d74aa5789e0d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:39.337485 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbc13b-aaa8-4257-94ff-d74aa5789e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:39.356545 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-c9bbc13b-aaa8-4257-94ff-d74aa5789e0d None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:39.357806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c9bbc13b-aaa8-4257-94ff-d74aa5789e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:39.376959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:39.424482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1205/2367] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:55:39.432130 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a270dd-44d1-4b00-8042-ea1adc729d02 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:39.432416 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a270dd-44d1-4b00-8042-ea1adc729d02 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:39.432692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a270dd-44d1-4b00-8042-ea1adc729d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:39.452311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-e3a270dd-44d1-4b00-8042-ea1adc729d02 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:39.453546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e3a270dd-44d1-4b00-8042-ea1adc729d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:39.462715 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1163/2368] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:55:39.467121 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbc13b-aaa8-4257-94ff-d74aa5789e0d None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:39.478370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbc13b-aaa8-4257-94ff-d74aa5789e0d None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:39.483243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d292fd-99bc-4011-93b0-2037fba0b847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:39.483945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d292fd-99bc-4011-93b0-2037fba0b847 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:39.484231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d292fd-99bc-4011-93b0-2037fba0b847 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:39.484687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d292fd-99bc-4011-93b0-2037fba0b847 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:39.485427 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d292fd-99bc-4011-93b0-2037fba0b847 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:55:39.485735 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d292fd-99bc-4011-93b0-2037fba0b847 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:39.487650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d292fd-99bc-4011-93b0-2037fba0b847 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:39.503545 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1206/2369] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:55:39.504884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a270dd-44d1-4b00-8042-ea1adc729d02 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:39.515334 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a270dd-44d1-4b00-8042-ea1adc729d02 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:39.520175 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae971c58-0006-4990-b8e7-6f4481917ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:39.520863 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae971c58-0006-4990-b8e7-6f4481917ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:39.521172 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae971c58-0006-4990-b8e7-6f4481917ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:39.522076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae971c58-0006-4990-b8e7-6f4481917ec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:39.523109 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae971c58-0006-4990-b8e7-6f4481917ec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:39.524936 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae971c58-0006-4990-b8e7-6f4481917ec6 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:39.528559 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:39.560775 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1164/2370] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:55:39 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:55:39.562007 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d292fd-99bc-4011-93b0-2037fba0b847 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:39.572744 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d292fd-99bc-4011-93b0-2037fba0b847 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:39.637431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e15c34-aa29-415f-8e7d-c897fa350eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:39.638158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e15c34-aa29-415f-8e7d-c897fa350eee admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:39.638443 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e15c34-aa29-415f-8e7d-c897fa350eee admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:39.638721 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e15c34-aa29-415f-8e7d-c897fa350eee admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:39.639442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e15c34-aa29-415f-8e7d-c897fa350eee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:39.641457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e15c34-aa29-415f-8e7d-c897fa350eee admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:39.662902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1207/2371] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:55:39 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:55:39.667256 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae971c58-0006-4990-b8e7-6f4481917ec6 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:39.678470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae971c58-0006-4990-b8e7-6f4481917ec6 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:39.683303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb37e0-e79d-470b-9e6d-c128b39cb72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:39.684022 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb37e0-e79d-470b-9e6d-c128b39cb72e admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:39.684312 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb37e0-e79d-470b-9e6d-c128b39cb72e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:39.684588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb37e0-e79d-470b-9e6d-c128b39cb72e admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:39.685341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb37e0-e79d-470b-9e6d-c128b39cb72e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:39.686926 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb37e0-e79d-470b-9e6d-c128b39cb72e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:39.698549 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1165/2372] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:55:39.702517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e15c34-aa29-415f-8e7d-c897fa350eee admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:39.716174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e15c34-aa29-415f-8e7d-c897fa350eee admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:39.722654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01258c1e-96b9-4436-8a19-30f8bcd5e931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:39.722654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01258c1e-96b9-4436-8a19-30f8bcd5e931 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:39.722654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01258c1e-96b9-4436-8a19-30f8bcd5e931 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:39.722654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01258c1e-96b9-4436-8a19-30f8bcd5e931 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:39.723475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01258c1e-96b9-4436-8a19-30f8bcd5e931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:39.726024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01258c1e-96b9-4436-8a19-30f8bcd5e931 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:39.745860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1208/2373] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:55:39.749237 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb37e0-e79d-470b-9e6d-c128b39cb72e admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:39.759484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb37e0-e79d-470b-9e6d-c128b39cb72e admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:39.769398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51350ad-dfff-4ff1-b889-521361f0cff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:39.770322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51350ad-dfff-4ff1-b889-521361f0cff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:39.770586 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51350ad-dfff-4ff1-b889-521361f0cff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:39.770887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51350ad-dfff-4ff1-b889-521361f0cff1 admin admin] PATH_INFO: `/v3/projects/98e1d76644d94fd9ada9ff1ae3267bf8/users/cc44212b992e4358bb37a1f8c4a6a77c/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:39.786401 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51350ad-dfff-4ff1-b889-521361f0cff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98e1d76644d94fd9ada9ff1ae3267bf8, user_id=cc44212b992e4358bb37a1f8c4a6a77c, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:39.789964 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51350ad-dfff-4ff1-b889-521361f0cff1 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:39.805630 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1166/2374] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:55:39 2022] PUT /identity/v3/projects/98e1d76644d94fd9ada9ff1ae3267bf8/users/cc44212b992e4358bb37a1f8c4a6a77c/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:55:39.810876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdda6cc-928e-429b-8f5f-a6bd8ac5738a None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:39.811185 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdda6cc-928e-429b-8f5f-a6bd8ac5738a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:39.811453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdda6cc-928e-429b-8f5f-a6bd8ac5738a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:39.830780 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-cfdda6cc-928e-429b-8f5f-a6bd8ac5738a None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:39.832101 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-cfdda6cc-928e-429b-8f5f-a6bd8ac5738a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:39.895243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1209/2375] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:55:39.905683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51350ad-dfff-4ff1-b889-521361f0cff1 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:39.907418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c51350ad-dfff-4ff1-b889-521361f0cff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:39.966451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51350ad-dfff-4ff1-b889-521361f0cff1 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:39.970207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b4f8a-7c90-4b0a-8c7e-a7577ae24f4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:39.970862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b4f8a-7c90-4b0a-8c7e-a7577ae24f4f service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:39.971151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b4f8a-7c90-4b0a-8c7e-a7577ae24f4f service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:39.971427 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b4f8a-7c90-4b0a-8c7e-a7577ae24f4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:39.981828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b4f8a-7c90-4b0a-8c7e-a7577ae24f4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:39.983312 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b4f8a-7c90-4b0a-8c7e-a7577ae24f4f service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:39.994998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1167/2376] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:55:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:55:40.932407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9717a9-b684-4335-8a9d-fbdbaa0d7334 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:40.932990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9717a9-b684-4335-8a9d-fbdbaa0d7334 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:40.932990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9717a9-b684-4335-8a9d-fbdbaa0d7334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:40.961177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-1f9717a9-b684-4335-8a9d-fbdbaa0d7334 None None] MFA Rules not processed for user `cc44212b992e4358bb37a1f8c4a6a77c`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:40.962421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1f9717a9-b684-4335-8a9d-fbdbaa0d7334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:40.981160 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:41.036730 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1210/2377] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:55:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:55:41.048367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b4f8a-7c90-4b0a-8c7e-a7577ae24f4f service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:41.050675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-909b4f8a-7c90-4b0a-8c7e-a7577ae24f4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:41.112260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b4f8a-7c90-4b0a-8c7e-a7577ae24f4f service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:41.116811 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c436db2-0f15-440b-a778-ca450fc2d966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:41.117463 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c436db2-0f15-440b-a778-ca450fc2d966 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:41.117743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c436db2-0f15-440b-a778-ca450fc2d966 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:41.118016 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c436db2-0f15-440b-a778-ca450fc2d966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:41.126857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c436db2-0f15-440b-a778-ca450fc2d966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:41.128325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c436db2-0f15-440b-a778-ca450fc2d966 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:41.137419 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1168/2378] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:55:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:55:49.972101 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e3081-33d8-484c-b3d1-c7a582fa831b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:49.972580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e3081-33d8-484c-b3d1-c7a582fa831b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:49.972772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e3081-33d8-484c-b3d1-c7a582fa831b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:49.999536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-f89e3081-33d8-484c-b3d1-c7a582fa831b None None] MFA Rules not processed for user `cc44212b992e4358bb37a1f8c4a6a77c`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:50.000775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f89e3081-33d8-484c-b3d1-c7a582fa831b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:50.008974 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1211/2379] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:55:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:55:51.133217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=73,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:53.522545 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147052d-f367-44d1-8130-1c879da3c232 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:53.523180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147052d-f367-44d1-8130-1c879da3c232 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:53.523180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147052d-f367-44d1-8130-1c879da3c232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:53.528815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e3081-33d8-484c-b3d1-c7a582fa831b None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:53.530397 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f89e3081-33d8-484c-b3d1-c7a582fa831b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:53.542922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-e147052d-f367-44d1-8130-1c879da3c232 None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:53.544304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e147052d-f367-44d1-8130-1c879da3c232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:53.563758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e3081-33d8-484c-b3d1-c7a582fa831b None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:53.585455 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e4349-1e9b-49f8-a670-83a4b19ca062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:53.586589 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e4349-1e9b-49f8-a670-83a4b19ca062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:53.587073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e4349-1e9b-49f8-a670-83a4b19ca062 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:53.587457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e4349-1e9b-49f8-a670-83a4b19ca062 admin admin] PATH_INFO: `/v3/users/7187dab7be7f437ebbd751d200b5f0c9` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:53.590276 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e4349-1e9b-49f8-a670-83a4b19ca062 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7187dab7be7f437ebbd751d200b5f0c9)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:53.592451 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e4349-1e9b-49f8-a670-83a4b19ca062 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:53.633609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1169/2380] 10.222.0.60 () {58 vars in 1009 bytes} [Thu Jul 21 20:55:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:55:53.693755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147052d-f367-44d1-8130-1c879da3c232 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:53.696830 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e147052d-f367-44d1-8130-1c879da3c232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:53.698040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-533e4349-1e9b-49f8-a670-83a4b19ca062 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7187dab7be7f437ebbd751d200b5f0c9'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:53.698882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-533e4349-1e9b-49f8-a670-83a4b19ca062 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7187dab7be7f437ebbd751d200b5f0c9'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:53.723593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-533e4349-1e9b-49f8-a670-83a4b19ca062 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7187dab7be7f437ebbd751d200b5f0c9'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:53.734926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-533e4349-1e9b-49f8-a670-83a4b19ca062 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7187dab7be7f437ebbd751d200b5f0c9'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:53.765064 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-533e4349-1e9b-49f8-a670-83a4b19ca062 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7187dab7be7f437ebbd751d200b5f0c9'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:53.778530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1212/2381] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:55:53 2022] DELETE /identity/v3/users/7187dab7be7f437ebbd751d200b5f0c9 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:55:53.782271 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ba2cf-c524-46fb-8e71-18ccccf704f5 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:53.782746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ba2cf-c524-46fb-8e71-18ccccf704f5 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:53.783179 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ba2cf-c524-46fb-8e71-18ccccf704f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:53.814106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d04ba2cf-c524-46fb-8e71-18ccccf704f5 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:53.817100 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d04ba2cf-c524-46fb-8e71-18ccccf704f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:53.863184 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147052d-f367-44d1-8130-1c879da3c232 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:53.897330 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1213/2382] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:55:53.951245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ba2cf-c524-46fb-8e71-18ccccf704f5 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:53.953750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d04ba2cf-c524-46fb-8e71-18ccccf704f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:54.009710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8bb5a-b2fe-4e66-8fd1-bdff6b9f856f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:54.009710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8bb5a-b2fe-4e66-8fd1-bdff6b9f856f service swift] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:54.009710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8bb5a-b2fe-4e66-8fd1-bdff6b9f856f service swift] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:54.009710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8bb5a-b2fe-4e66-8fd1-bdff6b9f856f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:54.016489 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6ff8bb5a-b2fe-4e66-8fd1-bdff6b9f856f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:54.044447 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:54.097287 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ba2cf-c524-46fb-8e71-18ccccf704f5 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:54.108951 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b7825-bcf3-4c07-8b2a-7bb6f94658ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:54.109751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b7825-bcf3-4c07-8b2a-7bb6f94658ef service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:54.111359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b7825-bcf3-4c07-8b2a-7bb6f94658ef service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:54.111696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b7825-bcf3-4c07-8b2a-7bb6f94658ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:54.152694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b7825-bcf3-4c07-8b2a-7bb6f94658ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:54.152988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b7825-bcf3-4c07-8b2a-7bb6f94658ef service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:54.175236 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1214/2383] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:55:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:55:54.204202 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:54.224423 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8bb5a-b2fe-4e66-8fd1-bdff6b9f856f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:54.233855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8bb5a-b2fe-4e66-8fd1-bdff6b9f856f service swift] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:54.246725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1170/2384] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 20:55:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 20:55:54.823624 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b7825-bcf3-4c07-8b2a-7bb6f94658ef service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:54.825220 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d75b7825-bcf3-4c07-8b2a-7bb6f94658ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:54.840574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b7825-bcf3-4c07-8b2a-7bb6f94658ef service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:54.845960 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa9a32-e9a8-42d3-9df8-ce5d4ec223f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:54.847040 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa9a32-e9a8-42d3-9df8-ce5d4ec223f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:54.847470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa9a32-e9a8-42d3-9df8-ce5d4ec223f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:54.847923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa9a32-e9a8-42d3-9df8-ce5d4ec223f1 admin admin] PATH_INFO: `/v3/projects/e42314cf48944558a247b652c87a9b5d` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:54.849315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa9a32-e9a8-42d3-9df8-ce5d4ec223f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e42314cf48944558a247b652c87a9b5d)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:54.851988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa9a32-e9a8-42d3-9df8-ce5d4ec223f1 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:54.914266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-f8aa9a32-e9a8-42d3-9df8-ce5d4ec223f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e42314cf48944558a247b652c87a9b5d', 'request_id': 'req-f8aa9a32-e9a8-42d3-9df8-ce5d4ec223f1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:54.917598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b8ebb-3b9f-4cbd-8a8c-5476b04f1167 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:54.918091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b8ebb-3b9f-4cbd-8a8c-5476b04f1167 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:54.918494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b8ebb-3b9f-4cbd-8a8c-5476b04f1167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:54.922563 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-f8aa9a32-e9a8-42d3-9df8-ce5d4ec223f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e42314cf48944558a247b652c87a9b5d', 'request_id': 'req-f8aa9a32-e9a8-42d3-9df8-ce5d4ec223f1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:54.930956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-f8aa9a32-e9a8-42d3-9df8-ce5d4ec223f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e42314cf48944558a247b652c87a9b5d', 'request_id': 'req-f8aa9a32-e9a8-42d3-9df8-ce5d4ec223f1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:54.943080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-db9b8ebb-3b9f-4cbd-8a8c-5476b04f1167 None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:54.945639 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-db9b8ebb-3b9f-4cbd-8a8c-5476b04f1167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:54.950215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-f8aa9a32-e9a8-42d3-9df8-ce5d4ec223f1 admin admin] The token cache is being invalidate because project e42314cf48944558a247b652c87a9b5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:55:54.950522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-f8aa9a32-e9a8-42d3-9df8-ce5d4ec223f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:54.951406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-f8aa9a32-e9a8-42d3-9df8-ce5d4ec223f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:54.953669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1215/2385] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:55:54 2022] DELETE /identity/v3/projects/e42314cf48944558a247b652c87a9b5d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:55:54.959638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a53c9f-b04b-4815-90cf-eddf37a3644a None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:54.959978 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a53c9f-b04b-4815-90cf-eddf37a3644a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:54.960321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a53c9f-b04b-4815-90cf-eddf37a3644a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:54.980219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-66a53c9f-b04b-4815-90cf-eddf37a3644a None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:54.981456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-66a53c9f-b04b-4815-90cf-eddf37a3644a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:55.026433 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1171/2386] 10.222.0.60 () {58 vars in 967 bytes} [Thu Jul 21 20:55:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:55:55.043814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b8ebb-3b9f-4cbd-8a8c-5476b04f1167 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:55.045374 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-db9b8ebb-3b9f-4cbd-8a8c-5476b04f1167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:55.070405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1216/2387] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:55:55.071735 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a53c9f-b04b-4815-90cf-eddf37a3644a None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:55.077756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-66a53c9f-b04b-4815-90cf-eddf37a3644a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:55.094158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a53c9f-b04b-4815-90cf-eddf37a3644a None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:55.098942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c794a-7fd3-4b0f-b91b-62fa4992da04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:55.099958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c794a-7fd3-4b0f-b91b-62fa4992da04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:55.100447 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c794a-7fd3-4b0f-b91b-62fa4992da04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:55.100886 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c794a-7fd3-4b0f-b91b-62fa4992da04 admin admin] PATH_INFO: `/v3/users/ec5dffdb634746548049eb6caa707d74` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:55.102235 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c794a-7fd3-4b0f-b91b-62fa4992da04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec5dffdb634746548049eb6caa707d74)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:55.104230 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c794a-7fd3-4b0f-b91b-62fa4992da04 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:55.134638 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b8ebb-3b9f-4cbd-8a8c-5476b04f1167 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:55.140404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af199705-4a42-4ca2-955c-99fe054ba329 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:55.141119 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af199705-4a42-4ca2-955c-99fe054ba329 service swift] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:55.141409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af199705-4a42-4ca2-955c-99fe054ba329 service swift] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:55.141698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af199705-4a42-4ca2-955c-99fe054ba329 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:55.150647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af199705-4a42-4ca2-955c-99fe054ba329 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:55.151447 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:55.152364 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af199705-4a42-4ca2-955c-99fe054ba329 service swift] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:55.153812 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b71c794a-7fd3-4b0f-b91b-62fa4992da04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec5dffdb634746548049eb6caa707d74'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:55.154800 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b71c794a-7fd3-4b0f-b91b-62fa4992da04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec5dffdb634746548049eb6caa707d74'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:55.161460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1172/2388] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 20:55:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:55:55.162705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103210ad-75f6-4aeb-ac40-b957b4ca2499 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:55.163194 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103210ad-75f6-4aeb-ac40-b957b4ca2499 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:55.163540 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103210ad-75f6-4aeb-ac40-b957b4ca2499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:55.173108 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b71c794a-7fd3-4b0f-b91b-62fa4992da04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec5dffdb634746548049eb6caa707d74'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:55.183797 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b71c794a-7fd3-4b0f-b91b-62fa4992da04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec5dffdb634746548049eb6caa707d74'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:55.189007 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-103210ad-75f6-4aeb-ac40-b957b4ca2499 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:55.190216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-103210ad-75f6-4aeb-ac40-b957b4ca2499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:55.191601 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b71c794a-7fd3-4b0f-b91b-62fa4992da04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec5dffdb634746548049eb6caa707d74'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:55.200253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1173/2389] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:55:55.202663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1217/2390] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:55:55 2022] DELETE /identity/v3/users/ec5dffdb634746548049eb6caa707d74 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:55:55.204582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103210ad-75f6-4aeb-ac40-b957b4ca2499 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:55.208272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e965bf-61f9-4bc9-9093-c9bda16273dd None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:55.208580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e965bf-61f9-4bc9-9093-c9bda16273dd None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:55.208866 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e965bf-61f9-4bc9-9093-c9bda16273dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:55.215080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103210ad-75f6-4aeb-ac40-b957b4ca2499 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:55.231571 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-a1e965bf-61f9-4bc9-9093-c9bda16273dd None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:55.232848 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a1e965bf-61f9-4bc9-9093-c9bda16273dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:55.298984 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5b124-62b7-4a74-9877-f440ae149e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:55.299677 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5b124-62b7-4a74-9877-f440ae149e74 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:55.300006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5b124-62b7-4a74-9877-f440ae149e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:55.300276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5b124-62b7-4a74-9877-f440ae149e74 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:55.301353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5b124-62b7-4a74-9877-f440ae149e74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:55:55.301849 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5b124-62b7-4a74-9877-f440ae149e74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:55.304174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5b124-62b7-4a74-9877-f440ae149e74 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:55.321775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1218/2391] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:55:55.325358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1174/2392] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:55:55.329388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e965bf-61f9-4bc9-9093-c9bda16273dd None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:55.332663 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5b124-62b7-4a74-9877-f440ae149e74 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:55.336015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-13b5b124-62b7-4a74-9877-f440ae149e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:55.343156 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e965bf-61f9-4bc9-9093-c9bda16273dd None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:55.348378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9304f9-e7c8-491f-af39-b5b92eabd2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:55.349048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9304f9-e7c8-491f-af39-b5b92eabd2d7 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:55.349335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9304f9-e7c8-491f-af39-b5b92eabd2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:55.350395 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9304f9-e7c8-491f-af39-b5b92eabd2d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:55.351297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9304f9-e7c8-491f-af39-b5b92eabd2d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:55.354037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9304f9-e7c8-491f-af39-b5b92eabd2d7 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:55.359606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:55.392707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1219/2393] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:55:55 2022] POST /identity/v3/projects => generated 391 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:55:55.397682 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9304f9-e7c8-491f-af39-b5b92eabd2d7 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:55.429667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5b124-62b7-4a74-9877-f440ae149e74 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:55.479644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9304f9-e7c8-491f-af39-b5b92eabd2d7 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:55.484478 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270f4d8-5d68-421c-81de-fab2c9f1cb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:55.485171 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270f4d8-5d68-421c-81de-fab2c9f1cb19 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:55.485458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270f4d8-5d68-421c-81de-fab2c9f1cb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:55.485731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270f4d8-5d68-421c-81de-fab2c9f1cb19 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:55.486688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270f4d8-5d68-421c-81de-fab2c9f1cb19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:55.487252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5993a00-db8e-4c1f-abe5-c51b7ab17526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:55.487927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5993a00-db8e-4c1f-abe5-c51b7ab17526 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:55.488255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5993a00-db8e-4c1f-abe5-c51b7ab17526 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:55.488584 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5993a00-db8e-4c1f-abe5-c51b7ab17526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:55.488681 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270f4d8-5d68-421c-81de-fab2c9f1cb19 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:55.498182 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5993a00-db8e-4c1f-abe5-c51b7ab17526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:55.499605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5993a00-db8e-4c1f-abe5-c51b7ab17526 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:55.510453 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1175/2394] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:55:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:55:55.511731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e656c9a-03d4-494e-ae41-d77af2fad2a3 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:55.511996 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1220/2395] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:55:55 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:55:55.515125 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e656c9a-03d4-494e-ae41-d77af2fad2a3 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:55.515424 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e656c9a-03d4-494e-ae41-d77af2fad2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:55.515912 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270f4d8-5d68-421c-81de-fab2c9f1cb19 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:55.528109 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270f4d8-5d68-421c-81de-fab2c9f1cb19 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:55.534615 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-8e656c9a-03d4-494e-ae41-d77af2fad2a3 None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:55.535844 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8e656c9a-03d4-494e-ae41-d77af2fad2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:55.536030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164bd6bb-6ee5-4dde-9798-5c05a1b79c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:55.536699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bd6bb-6ee5-4dde-9798-5c05a1b79c04 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:55.537102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bd6bb-6ee5-4dde-9798-5c05a1b79c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:55.537542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bd6bb-6ee5-4dde-9798-5c05a1b79c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:55.538654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164bd6bb-6ee5-4dde-9798-5c05a1b79c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:55.540405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164bd6bb-6ee5-4dde-9798-5c05a1b79c04 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:55.551991 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1221/2396] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:55:55.556917 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164bd6bb-6ee5-4dde-9798-5c05a1b79c04 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:55.567158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164bd6bb-6ee5-4dde-9798-5c05a1b79c04 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:55.582708 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076c5ee-2927-42ca-96a6-ff8d15033b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:55.583438 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076c5ee-2927-42ca-96a6-ff8d15033b9e admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:55.583682 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076c5ee-2927-42ca-96a6-ff8d15033b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:55.584072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076c5ee-2927-42ca-96a6-ff8d15033b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:55.585696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076c5ee-2927-42ca-96a6-ff8d15033b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:55.587922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076c5ee-2927-42ca-96a6-ff8d15033b9e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:55.601371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1222/2397] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:55:55.605691 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076c5ee-2927-42ca-96a6-ff8d15033b9e admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:55.618409 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076c5ee-2927-42ca-96a6-ff8d15033b9e admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:55.620556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1176/2398] 10.222.0.60 () {58 vars in 1009 bytes} [Thu Jul 21 20:55:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:55:55.624425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37f15f-011f-4bf1-a869-8a44f6600231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:55.625324 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37f15f-011f-4bf1-a869-8a44f6600231 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:55.625606 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37f15f-011f-4bf1-a869-8a44f6600231 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:55.625925 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37f15f-011f-4bf1-a869-8a44f6600231 admin admin] PATH_INFO: `/v3/projects/16a5186ab14a4f4e9f7333cbcb66a07a/users/a40b5e463cbc4581a728f36235795e47/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:55.632928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:55.637491 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e656c9a-03d4-494e-ae41-d77af2fad2a3 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:55.639069 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8e656c9a-03d4-494e-ae41-d77af2fad2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:55.649498 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37f15f-011f-4bf1-a869-8a44f6600231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16a5186ab14a4f4e9f7333cbcb66a07a, user_id=a40b5e463cbc4581a728f36235795e47, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:55.652734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37f15f-011f-4bf1-a869-8a44f6600231 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:55.668688 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1223/2399] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:55:55 2022] PUT /identity/v3/projects/16a5186ab14a4f4e9f7333cbcb66a07a/users/a40b5e463cbc4581a728f36235795e47/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:55:55.680995 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff4661-a7a6-470f-a900-8b62f6b2fb5a None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:55.681335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff4661-a7a6-470f-a900-8b62f6b2fb5a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:55.681810 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff4661-a7a6-470f-a900-8b62f6b2fb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:55.713118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-64ff4661-a7a6-470f-a900-8b62f6b2fb5a None None] MFA Rules not processed for user `a40b5e463cbc4581a728f36235795e47`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:55.714434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-64ff4661-a7a6-470f-a900-8b62f6b2fb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:55.736852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e656c9a-03d4-494e-ae41-d77af2fad2a3 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:55.757690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:55.807794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1224/2400] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:55:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:55:55.821347 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff4661-a7a6-470f-a900-8b62f6b2fb5a None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:55.822752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-64ff4661-a7a6-470f-a900-8b62f6b2fb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:55.823799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060621cf-4a7f-46e9-adf3-bf848694b0b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:55.824988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060621cf-4a7f-46e9-adf3-bf848694b0b2 service swift] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:55.825507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060621cf-4a7f-46e9-adf3-bf848694b0b2 service swift] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:55.826957 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060621cf-4a7f-46e9-adf3-bf848694b0b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:55.888428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff4661-a7a6-470f-a900-8b62f6b2fb5a None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:55.892608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069686d-5627-4c32-bebf-3937fd57af4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:55.893333 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069686d-5627-4c32-bebf-3937fd57af4d service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:55.893685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069686d-5627-4c32-bebf-3937fd57af4d service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:55.894029 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069686d-5627-4c32-bebf-3937fd57af4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:55.903583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060621cf-4a7f-46e9-adf3-bf848694b0b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:55.905303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060621cf-4a7f-46e9-adf3-bf848694b0b2 service swift] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:55.906798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069686d-5627-4c32-bebf-3937fd57af4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:55.908314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069686d-5627-4c32-bebf-3937fd57af4d service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:55.915520 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1177/2401] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 20:55:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:55:55.920488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060621cf-4a7f-46e9-adf3-bf848694b0b2 service swift] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:55.921041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1225/2402] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:55:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:55:55.928912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-060621cf-4a7f-46e9-adf3-bf848694b0b2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:56.018569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060621cf-4a7f-46e9-adf3-bf848694b0b2 service swift] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:56.024945 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e36696-8f40-450d-a7fd-56ff048b8c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:56.026060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e36696-8f40-450d-a7fd-56ff048b8c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:56.026402 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e36696-8f40-450d-a7fd-56ff048b8c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:56.026755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e36696-8f40-450d-a7fd-56ff048b8c70 admin admin] PATH_INFO: `/v3/projects/b7497b0c2c9c42dda38332bb368e56bb` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:56.028280 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e36696-8f40-450d-a7fd-56ff048b8c70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7497b0c2c9c42dda38332bb368e56bb)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:56.030945 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e36696-8f40-450d-a7fd-56ff048b8c70 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:56.095892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d6e36696-8f40-450d-a7fd-56ff048b8c70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7497b0c2c9c42dda38332bb368e56bb', 'request_id': 'req-d6e36696-8f40-450d-a7fd-56ff048b8c70'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:56.106179 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d6e36696-8f40-450d-a7fd-56ff048b8c70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7497b0c2c9c42dda38332bb368e56bb', 'request_id': 'req-d6e36696-8f40-450d-a7fd-56ff048b8c70'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:56.114390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:56.117597 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d6e36696-8f40-450d-a7fd-56ff048b8c70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7497b0c2c9c42dda38332bb368e56bb', 'request_id': 'req-d6e36696-8f40-450d-a7fd-56ff048b8c70'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:56.135329 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d6e36696-8f40-450d-a7fd-56ff048b8c70 admin admin] The token cache is being invalidate because project b7497b0c2c9c42dda38332bb368e56bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:55:56.135692 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d6e36696-8f40-450d-a7fd-56ff048b8c70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:56.136561 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d6e36696-8f40-450d-a7fd-56ff048b8c70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:55:56.139017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1178/2403] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:55:55 2022] DELETE /identity/v3/projects/b7497b0c2c9c42dda38332bb368e56bb => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:55:56.144389 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feeee33-a0d5-4a9a-b709-25d36672ccc2 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:56.144707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feeee33-a0d5-4a9a-b709-25d36672ccc2 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:56.144989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feeee33-a0d5-4a9a-b709-25d36672ccc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:56.166094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-3feeee33-a0d5-4a9a-b709-25d36672ccc2 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:56.167452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-3feeee33-a0d5-4a9a-b709-25d36672ccc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:56.248755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1226/2404] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:55:56.257726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babecc7-3151-4a58-89d9-39f3d614f835 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:56.258020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babecc7-3151-4a58-89d9-39f3d614f835 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:56.258306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babecc7-3151-4a58-89d9-39f3d614f835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:56.286504 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-7babecc7-3151-4a58-89d9-39f3d614f835 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:56.288191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7babecc7-3151-4a58-89d9-39f3d614f835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:56.299763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1179/2405] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:55:56.301420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feeee33-a0d5-4a9a-b709-25d36672ccc2 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:56.315527 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feeee33-a0d5-4a9a-b709-25d36672ccc2 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:56.320888 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d795b3-4a06-4a38-997c-b9264df87309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:56.321553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d795b3-4a06-4a38-997c-b9264df87309 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:56.321882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d795b3-4a06-4a38-997c-b9264df87309 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:56.322116 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d795b3-4a06-4a38-997c-b9264df87309 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:56.322826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d795b3-4a06-4a38-997c-b9264df87309 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:55:56.323129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d795b3-4a06-4a38-997c-b9264df87309 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:56.325657 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d795b3-4a06-4a38-997c-b9264df87309 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:56.339982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1227/2406] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:55:56.344417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babecc7-3151-4a58-89d9-39f3d614f835 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:56.349021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16a8e9-53b6-455b-9b4c-8ecd38b95fd0 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:56.349403 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16a8e9-53b6-455b-9b4c-8ecd38b95fd0 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:56.350470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16a8e9-53b6-455b-9b4c-8ecd38b95fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:56.354755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babecc7-3151-4a58-89d9-39f3d614f835 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:56.358572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:56.362587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b8c32-fa81-4157-bacd-7676587198b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:56.363808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b8c32-fa81-4157-bacd-7676587198b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:56.364331 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b8c32-fa81-4157-bacd-7676587198b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:56.364843 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b8c32-fa81-4157-bacd-7676587198b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:56.366664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b8c32-fa81-4157-bacd-7676587198b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:56.369038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b8c32-fa81-4157-bacd-7676587198b5 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:56.372464 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-fe16a8e9-53b6-455b-9b4c-8ecd38b95fd0 None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:56.373683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fe16a8e9-53b6-455b-9b4c-8ecd38b95fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:56.398369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1180/2407] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:55:56 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:55:56.404845 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b8c32-fa81-4157-bacd-7676587198b5 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:56.434526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1228/2408] 10.222.0.60 () {58 vars in 967 bytes} [Thu Jul 21 20:55:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:55:56.448699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16a8e9-53b6-455b-9b4c-8ecd38b95fd0 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:56.450128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fe16a8e9-53b6-455b-9b4c-8ecd38b95fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:56.479371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b8c32-fa81-4157-bacd-7676587198b5 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:56.484200 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1afddb-aa98-4088-84b3-a7620c9b4424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:56.484869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1afddb-aa98-4088-84b3-a7620c9b4424 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:56.485148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1afddb-aa98-4088-84b3-a7620c9b4424 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:56.485422 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1afddb-aa98-4088-84b3-a7620c9b4424 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:56.486765 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1afddb-aa98-4088-84b3-a7620c9b4424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:56.488300 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1afddb-aa98-4088-84b3-a7620c9b4424 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:56.516582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1181/2409] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:55:56 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:55:56.517922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1afddb-aa98-4088-84b3-a7620c9b4424 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:56.529908 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1afddb-aa98-4088-84b3-a7620c9b4424 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:56.534968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade485d-9abf-435b-b73b-46f3e347cd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:56.535764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade485d-9abf-435b-b73b-46f3e347cd19 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:56.536142 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade485d-9abf-435b-b73b-46f3e347cd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:56.536449 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade485d-9abf-435b-b73b-46f3e347cd19 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:56.537774 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade485d-9abf-435b-b73b-46f3e347cd19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:56.539667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade485d-9abf-435b-b73b-46f3e347cd19 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:56.548798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16a8e9-53b6-455b-9b4c-8ecd38b95fd0 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:56.558287 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1182/2410] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:55:56.559694 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade485d-9abf-435b-b73b-46f3e347cd19 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:56.570695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade485d-9abf-435b-b73b-46f3e347cd19 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:56.576017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6472ad9-ba91-46db-8722-7b33b6439475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:56.576713 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6472ad9-ba91-46db-8722-7b33b6439475 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:56.576999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6472ad9-ba91-46db-8722-7b33b6439475 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:56.577278 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6472ad9-ba91-46db-8722-7b33b6439475 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:56.578037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6472ad9-ba91-46db-8722-7b33b6439475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:56.580023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6472ad9-ba91-46db-8722-7b33b6439475 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:56.597498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1183/2411] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:55:56.597855 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6472ad9-ba91-46db-8722-7b33b6439475 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:56.609888 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6472ad9-ba91-46db-8722-7b33b6439475 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:56.615436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20e918-b5eb-448a-baf5-53cad65a0beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:56.616384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20e918-b5eb-448a-baf5-53cad65a0beb admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:56.616667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20e918-b5eb-448a-baf5-53cad65a0beb admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:56.617128 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20e918-b5eb-448a-baf5-53cad65a0beb admin admin] PATH_INFO: `/v3/projects/3dd1eb8097fd42bfa076ac2719a62039/users/9ef7004aeef04a0f8376c460bab263d3/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:56.635999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20e918-b5eb-448a-baf5-53cad65a0beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dd1eb8097fd42bfa076ac2719a62039, user_id=9ef7004aeef04a0f8376c460bab263d3, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:56.639150 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20e918-b5eb-448a-baf5-53cad65a0beb admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:56.640518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089a06d-8972-4655-9db5-c8a989c24bc1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:56.641455 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089a06d-8972-4655-9db5-c8a989c24bc1 service swift] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:56.641992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089a06d-8972-4655-9db5-c8a989c24bc1 service swift] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:56.642616 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089a06d-8972-4655-9db5-c8a989c24bc1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:56.657800 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1184/2412] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:55:56 2022] PUT /identity/v3/projects/3dd1eb8097fd42bfa076ac2719a62039/users/9ef7004aeef04a0f8376c460bab263d3/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:55:56.728178 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:56.729561 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddc8fc-5b25-4bb7-9220-13e27258461f None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:56.729850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddc8fc-5b25-4bb7-9220-13e27258461f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:56.730120 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddc8fc-5b25-4bb7-9220-13e27258461f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:56.731395 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089a06d-8972-4655-9db5-c8a989c24bc1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:56.733114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089a06d-8972-4655-9db5-c8a989c24bc1 service swift] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:56.743821 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1229/2413] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 20:55:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 20:55:56.764614 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-16ddc8fc-5b25-4bb7-9220-13e27258461f None None] MFA Rules not processed for user `9ef7004aeef04a0f8376c460bab263d3`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:55:56.766027 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-16ddc8fc-5b25-4bb7-9220-13e27258461f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:56.828856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:55:56.852175 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1185/2414] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:55:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:55:56.862816 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089a06d-8972-4655-9db5-c8a989c24bc1 service swift] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:55:56.864333 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-3089a06d-8972-4655-9db5-c8a989c24bc1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:55:56.941791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089a06d-8972-4655-9db5-c8a989c24bc1 service swift] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:55:57.041541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de8209-41aa-4a10-b7cc-8cb9aa661d7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:55:57.042208 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de8209-41aa-4a10-b7cc-8cb9aa661d7e service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:55:57.042490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de8209-41aa-4a10-b7cc-8cb9aa661d7e service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:55:57.042764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de8209-41aa-4a10-b7cc-8cb9aa661d7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:55:57.056468 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de8209-41aa-4a10-b7cc-8cb9aa661d7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:55:57.057946 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de8209-41aa-4a10-b7cc-8cb9aa661d7e service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:55:57.070241 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1230/2415] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:55:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:56:05.436991 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ddc8fc-5b25-4bb7-9220-13e27258461f None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:05.438467 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-16ddc8fc-5b25-4bb7-9220-13e27258461f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:05.525390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ddc8fc-5b25-4bb7-9220-13e27258461f None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:05.530685 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181686f-1da4-490b-b0d2-f6171832bb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:05.531770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181686f-1da4-490b-b0d2-f6171832bb84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:05.531992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181686f-1da4-490b-b0d2-f6171832bb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:05.532319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181686f-1da4-490b-b0d2-f6171832bb84 admin admin] PATH_INFO: `/v3/users/5c6e45e0cb81448baaae7d4f2ccd5d28` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:05.533482 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181686f-1da4-490b-b0d2-f6171832bb84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c6e45e0cb81448baaae7d4f2ccd5d28)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:05.535664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181686f-1da4-490b-b0d2-f6171832bb84 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:05.594135 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2181686f-1da4-490b-b0d2-f6171832bb84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c6e45e0cb81448baaae7d4f2ccd5d28'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:05.604023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2181686f-1da4-490b-b0d2-f6171832bb84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c6e45e0cb81448baaae7d4f2ccd5d28'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:05.604824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2181686f-1da4-490b-b0d2-f6171832bb84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c6e45e0cb81448baaae7d4f2ccd5d28'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:05.614315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2181686f-1da4-490b-b0d2-f6171832bb84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c6e45e0cb81448baaae7d4f2ccd5d28'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:05.615199 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2181686f-1da4-490b-b0d2-f6171832bb84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c6e45e0cb81448baaae7d4f2ccd5d28'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:05.655275 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1186/2416] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:56:05 2022] DELETE /identity/v3/users/5c6e45e0cb81448baaae7d4f2ccd5d28 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:56:05.657270 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c3722-161a-42ee-a948-f90f361c0381 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:05.657758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c3722-161a-42ee-a948-f90f361c0381 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:05.657866 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c3722-161a-42ee-a948-f90f361c0381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:05.682201 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-447c3722-161a-42ee-a948-f90f361c0381 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:56:05.683455 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-447c3722-161a-42ee-a948-f90f361c0381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:05.758456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1231/2417] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:56:05.768541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181686f-1da4-490b-b0d2-f6171832bb84 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:05.769991 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-2181686f-1da4-490b-b0d2-f6171832bb84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:05.834383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181686f-1da4-490b-b0d2-f6171832bb84 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:05.839533 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3aa6bb-c02c-489c-acc5-241679d8787b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:05.840257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3aa6bb-c02c-489c-acc5-241679d8787b service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:05.840548 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3aa6bb-c02c-489c-acc5-241679d8787b service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:05.840822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3aa6bb-c02c-489c-acc5-241679d8787b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:05.851303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3aa6bb-c02c-489c-acc5-241679d8787b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:05.853047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3aa6bb-c02c-489c-acc5-241679d8787b service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:05.863316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1187/2418] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:56:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:56:06.046938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c3722-161a-42ee-a948-f90f361c0381 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:06.048563 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-447c3722-161a-42ee-a948-f90f361c0381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:06.062213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c3722-161a-42ee-a948-f90f361c0381 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:06.067388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa658df-3e53-4969-a150-24c7266f42ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:06.068715 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa658df-3e53-4969-a150-24c7266f42ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:06.069152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa658df-3e53-4969-a150-24c7266f42ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:06.069542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa658df-3e53-4969-a150-24c7266f42ec admin admin] PATH_INFO: `/v3/projects/a0f56ad01c544c958f8e65367ff5c575` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:06.073264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa658df-3e53-4969-a150-24c7266f42ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f56ad01c544c958f8e65367ff5c575)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:06.075325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa658df-3e53-4969-a150-24c7266f42ec admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:06.136284 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:56:06.137918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-daa658df-3e53-4969-a150-24c7266f42ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f56ad01c544c958f8e65367ff5c575', 'request_id': 'req-daa658df-3e53-4969-a150-24c7266f42ec'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:06.148986 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-daa658df-3e53-4969-a150-24c7266f42ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f56ad01c544c958f8e65367ff5c575', 'request_id': 'req-daa658df-3e53-4969-a150-24c7266f42ec'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:06.155425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-daa658df-3e53-4969-a150-24c7266f42ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f56ad01c544c958f8e65367ff5c575', 'request_id': 'req-daa658df-3e53-4969-a150-24c7266f42ec'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:06.171161 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-daa658df-3e53-4969-a150-24c7266f42ec admin admin] The token cache is being invalidate because project a0f56ad01c544c958f8e65367ff5c575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:56:06.171523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-daa658df-3e53-4969-a150-24c7266f42ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:06.176292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-daa658df-3e53-4969-a150-24c7266f42ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:06.178988 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1232/2419] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:56:06 2022] DELETE /identity/v3/projects/a0f56ad01c544c958f8e65367ff5c575 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:56:06.184275 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a303379-cd7c-4139-8323-2a22778dd513 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:06.184884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a303379-cd7c-4139-8323-2a22778dd513 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:06.185396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a303379-cd7c-4139-8323-2a22778dd513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:06.206981 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-6a303379-cd7c-4139-8323-2a22778dd513 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:56:06.208516 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6a303379-cd7c-4139-8323-2a22778dd513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:06.221026 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:56:06.280879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1188/2420] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:56:06.289695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ada1a4-c6f9-46a8-9003-631001e03258 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:06.289989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ada1a4-c6f9-46a8-9003-631001e03258 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:06.290273 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ada1a4-c6f9-46a8-9003-631001e03258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:06.310261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-17ada1a4-c6f9-46a8-9003-631001e03258 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:56:06.311483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-17ada1a4-c6f9-46a8-9003-631001e03258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:06.319746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1233/2421] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:56:06.323985 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a303379-cd7c-4139-8323-2a22778dd513 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:06.333755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a303379-cd7c-4139-8323-2a22778dd513 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:06.339177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816697fc-f2d8-43f2-a228-708016fa5c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:06.339884 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816697fc-f2d8-43f2-a228-708016fa5c80 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:06.340175 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816697fc-f2d8-43f2-a228-708016fa5c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:06.340583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816697fc-f2d8-43f2-a228-708016fa5c80 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:06.341328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816697fc-f2d8-43f2-a228-708016fa5c80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 20:56:06.341703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816697fc-f2d8-43f2-a228-708016fa5c80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:06.343316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816697fc-f2d8-43f2-a228-708016fa5c80 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:06.359850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1189/2422] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 20:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 20:56:06.360940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ada1a4-c6f9-46a8-9003-631001e03258 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:06.371000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ada1a4-c6f9-46a8-9003-631001e03258 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:06.375684 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b919f78-6eac-4f92-a249-7d77ea989065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:06.376364 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b919f78-6eac-4f92-a249-7d77ea989065 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:06.376669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b919f78-6eac-4f92-a249-7d77ea989065 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:06.376948 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b919f78-6eac-4f92-a249-7d77ea989065 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:06.377647 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b919f78-6eac-4f92-a249-7d77ea989065 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:06.379543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b919f78-6eac-4f92-a249-7d77ea989065 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:06.407546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1234/2423] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:56:06 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 20:56:06.411686 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816697fc-f2d8-43f2-a228-708016fa5c80 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:06.422970 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816697fc-f2d8-43f2-a228-708016fa5c80 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:06.484375 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f86cc-403a-4c11-9ace-e8f219583ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:06.485093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f86cc-403a-4c11-9ace-e8f219583ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:06.485326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f86cc-403a-4c11-9ace-e8f219583ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:06.486698 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f86cc-403a-4c11-9ace-e8f219583ab0 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:06.487821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f86cc-403a-4c11-9ace-e8f219583ab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:06.489760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f86cc-403a-4c11-9ace-e8f219583ab0 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:06.511412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1190/2424] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:56:06 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:56:06.515637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b919f78-6eac-4f92-a249-7d77ea989065 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:06.526973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b919f78-6eac-4f92-a249-7d77ea989065 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:06.531756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130939b-ccc0-45c7-9ffa-afe24235d7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:06.532425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130939b-ccc0-45c7-9ffa-afe24235d7ce admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:06.532930 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130939b-ccc0-45c7-9ffa-afe24235d7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:06.533218 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130939b-ccc0-45c7-9ffa-afe24235d7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:06.533948 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130939b-ccc0-45c7-9ffa-afe24235d7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:06.535539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130939b-ccc0-45c7-9ffa-afe24235d7ce admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:06.550549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1235/2425] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 20:56:06.551878 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f86cc-403a-4c11-9ace-e8f219583ab0 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:06.563196 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f86cc-403a-4c11-9ace-e8f219583ab0 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:06.574965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a8a4c-07cb-4386-8d10-967e25166a93 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:06.575255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a8a4c-07cb-4386-8d10-967e25166a93 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:06.575518 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc2a64-e292-48ce-9ea6-a9b011a388bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:06.575645 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a8a4c-07cb-4386-8d10-967e25166a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:06.576401 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc2a64-e292-48ce-9ea6-a9b011a388bc admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:06.576869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc2a64-e292-48ce-9ea6-a9b011a388bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:06.577150 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc2a64-e292-48ce-9ea6-a9b011a388bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:06.577962 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc2a64-e292-48ce-9ea6-a9b011a388bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:06.579643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc2a64-e292-48ce-9ea6-a9b011a388bc admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:06.592824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1191/2426] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:56:06.593512 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-176a8a4c-07cb-4386-8d10-967e25166a93 None None] MFA Rules not processed for user `24e4efb87cd34008a1b83c0ea0cce5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:56:06.594734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-176a8a4c-07cb-4386-8d10-967e25166a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:06.597071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc2a64-e292-48ce-9ea6-a9b011a388bc admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:06.608627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc2a64-e292-48ce-9ea6-a9b011a388bc admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:06.613725 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbb8b1-d8de-4ddd-ab59-886c8d77feba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:06.614612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbb8b1-d8de-4ddd-ab59-886c8d77feba admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:06.614901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbb8b1-d8de-4ddd-ab59-886c8d77feba admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:06.615178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbb8b1-d8de-4ddd-ab59-886c8d77feba admin admin] PATH_INFO: `/v3/projects/f2d1bc3b35054c44bbe955a9eddc3052/users/2b3617b8bcac4227b1295e294eec664a/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:06.633592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbb8b1-d8de-4ddd-ab59-886c8d77feba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2d1bc3b35054c44bbe955a9eddc3052, user_id=2b3617b8bcac4227b1295e294eec664a, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:06.636214 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbb8b1-d8de-4ddd-ab59-886c8d77feba admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:06.649798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1192/2427] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:56:06 2022] PUT /identity/v3/projects/f2d1bc3b35054c44bbe955a9eddc3052/users/2b3617b8bcac4227b1295e294eec664a/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 20:56:06.654767 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1236/2428] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:56:06.655216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e23e6-ff00-4f8c-b8b0-33852045b1ed None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:06.655511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e23e6-ff00-4f8c-b8b0-33852045b1ed None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:06.655834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e23e6-ff00-4f8c-b8b0-33852045b1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:06.672706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d11e23e6-ff00-4f8c-b8b0-33852045b1ed None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:56:06.674015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d11e23e6-ff00-4f8c-b8b0-33852045b1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:06.729233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:56:06.741821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1193/2429] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 20:56:06.752594 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a8a4c-07cb-4386-8d10-967e25166a93 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:06.754077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-176a8a4c-07cb-4386-8d10-967e25166a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:06.812209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a8a4c-07cb-4386-8d10-967e25166a93 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:06.864942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20728d-d3c1-41d5-a998-2316580df9dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:06.865676 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20728d-d3c1-41d5-a998-2316580df9dc service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:06.866059 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20728d-d3c1-41d5-a998-2316580df9dc service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:06.866902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20728d-d3c1-41d5-a998-2316580df9dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:06.876979 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20728d-d3c1-41d5-a998-2316580df9dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:06.878403 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20728d-d3c1-41d5-a998-2316580df9dc service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:06.884454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:56:06.890107 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1237/2430] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:56:11.363208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b6951-af70-495f-9f0a-b927a998eb82 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:11.363752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b6951-af70-495f-9f0a-b927a998eb82 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:11.363921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b6951-af70-495f-9f0a-b927a998eb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:11.425220 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c04b6951-af70-495f-9f0a-b927a998eb82 None None] MFA Rules not processed for user `2b3617b8bcac4227b1295e294eec664a`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:56:11.426789 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c04b6951-af70-495f-9f0a-b927a998eb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:11.517703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1194/2431] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:56:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:56:11.521752 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20728d-d3c1-41d5-a998-2316580df9dc service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:11.535379 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20728d-d3c1-41d5-a998-2316580df9dc service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:11.540132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4c12a-6f82-4ac5-888c-c9dd7f384b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:11.540985 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4c12a-6f82-4ac5-888c-c9dd7f384b38 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:11.541337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4c12a-6f82-4ac5-888c-c9dd7f384b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:11.541811 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4c12a-6f82-4ac5-888c-c9dd7f384b38 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:11.542454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4c12a-6f82-4ac5-888c-c9dd7f384b38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:11.548922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4c12a-6f82-4ac5-888c-c9dd7f384b38 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:11.599469 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1238/2432] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 20:56:11 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:56:11.600090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b6951-af70-495f-9f0a-b927a998eb82 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:11.634429 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b6951-af70-495f-9f0a-b927a998eb82 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:11.725402 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca5cd8-b7fd-48b6-82be-9042468b8d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:11.726408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca5cd8-b7fd-48b6-82be-9042468b8d48 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:11.726926 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca5cd8-b7fd-48b6-82be-9042468b8d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:11.727207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca5cd8-b7fd-48b6-82be-9042468b8d48 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:11.727950 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca5cd8-b7fd-48b6-82be-9042468b8d48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:11.729916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca5cd8-b7fd-48b6-82be-9042468b8d48 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:11.757219 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1195/2433] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 20:56:11 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 20:56:11.761574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4c12a-6f82-4ac5-888c-c9dd7f384b38 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:11.775963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4c12a-6f82-4ac5-888c-c9dd7f384b38 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:11.789603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1ed75-cae7-41a5-a420-04eac6a8f594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:11.791472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1ed75-cae7-41a5-a420-04eac6a8f594 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:11.791971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1ed75-cae7-41a5-a420-04eac6a8f594 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:11.792474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1ed75-cae7-41a5-a420-04eac6a8f594 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:11.793299 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1ed75-cae7-41a5-a420-04eac6a8f594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:11.795819 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1ed75-cae7-41a5-a420-04eac6a8f594 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:11.813657 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1239/2434] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:56:11.818416 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca5cd8-b7fd-48b6-82be-9042468b8d48 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:11.834589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca5cd8-b7fd-48b6-82be-9042468b8d48 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:11.859405 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499885d7-b89f-40de-b6cc-7010da19c2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:11.860468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499885d7-b89f-40de-b6cc-7010da19c2a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:11.860870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499885d7-b89f-40de-b6cc-7010da19c2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:11.861970 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499885d7-b89f-40de-b6cc-7010da19c2a4 admin admin] PATH_INFO: `/v3/projects/fec83747abfe4021a8b4336d17d0c214/users/fe45d7e58a064f69ba18ed69b47c2050/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:11.891036 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499885d7-b89f-40de-b6cc-7010da19c2a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec83747abfe4021a8b4336d17d0c214, user_id=fe45d7e58a064f69ba18ed69b47c2050, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:11.893982 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499885d7-b89f-40de-b6cc-7010da19c2a4 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:11.912028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1196/2435] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:56:11 2022] PUT /identity/v3/projects/fec83747abfe4021a8b4336d17d0c214/users/fe45d7e58a064f69ba18ed69b47c2050/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:56:11.916881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1ed75-cae7-41a5-a420-04eac6a8f594 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:11.937631 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1ed75-cae7-41a5-a420-04eac6a8f594 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:12.004142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d947d-520b-49e4-9071-3580ed2561b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:12.005077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d947d-520b-49e4-9071-3580ed2561b2 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:12.005521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d947d-520b-49e4-9071-3580ed2561b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:12.005956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d947d-520b-49e4-9071-3580ed2561b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:12.007451 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d947d-520b-49e4-9071-3580ed2561b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:12.009421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d947d-520b-49e4-9071-3580ed2561b2 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:12.027990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1240/2436] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 20:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 20:56:12.034066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499885d7-b89f-40de-b6cc-7010da19c2a4 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:12.058690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499885d7-b89f-40de-b6cc-7010da19c2a4 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:12.064172 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a4fe1-31f9-41d0-9e2f-b641703bb94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:12.065228 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a4fe1-31f9-41d0-9e2f-b641703bb94a admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:12.065605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a4fe1-31f9-41d0-9e2f-b641703bb94a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:12.065971 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a4fe1-31f9-41d0-9e2f-b641703bb94a admin admin] PATH_INFO: `/v3/projects/fec83747abfe4021a8b4336d17d0c214/users/fe45d7e58a064f69ba18ed69b47c2050/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:12.069282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a4fe1-31f9-41d0-9e2f-b641703bb94a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec83747abfe4021a8b4336d17d0c214, user_id=fe45d7e58a064f69ba18ed69b47c2050, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:12.072133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a4fe1-31f9-41d0-9e2f-b641703bb94a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:12.093327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1197/2437] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 20:56:12 2022] PUT /identity/v3/projects/fec83747abfe4021a8b4336d17d0c214/users/fe45d7e58a064f69ba18ed69b47c2050/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:56:18.905650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959289e3-e7a1-40e2-83d8-7e6223edbfef None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:18.906656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959289e3-e7a1-40e2-83d8-7e6223edbfef None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:18.907154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959289e3-e7a1-40e2-83d8-7e6223edbfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:18.941524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-959289e3-e7a1-40e2-83d8-7e6223edbfef None None] MFA Rules not processed for user `fe45d7e58a064f69ba18ed69b47c2050`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:56:18.943203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-959289e3-e7a1-40e2-83d8-7e6223edbfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:19.025468 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1241/2438] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:56:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:56:19.041136 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a4fe1-31f9-41d0-9e2f-b641703bb94a admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:19.042906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-192a4fe1-31f9-41d0-9e2f-b641703bb94a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:19.093473 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:56:19.147574 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a4fe1-31f9-41d0-9e2f-b641703bb94a admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:19.159262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722537f0-d9bb-431a-aff2-ff10dd9f67d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:19.161103 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722537f0-d9bb-431a-aff2-ff10dd9f67d0 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:19.161506 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722537f0-d9bb-431a-aff2-ff10dd9f67d0 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:19.161950 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722537f0-d9bb-431a-aff2-ff10dd9f67d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:19.266700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722537f0-d9bb-431a-aff2-ff10dd9f67d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:19.267992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722537f0-d9bb-431a-aff2-ff10dd9f67d0 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:19.282849 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1198/2439] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:56:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:56:20.304489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959289e3-e7a1-40e2-83d8-7e6223edbfef None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:20.308588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-959289e3-e7a1-40e2-83d8-7e6223edbfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:20.442193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:56:20.466502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959289e3-e7a1-40e2-83d8-7e6223edbfef None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:20.473693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca0812-c5aa-4a12-9fec-31d79b0e5c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:20.480812 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca0812-c5aa-4a12-9fec-31d79b0e5c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:20.481634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca0812-c5aa-4a12-9fec-31d79b0e5c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:20.482066 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca0812-c5aa-4a12-9fec-31d79b0e5c0e admin admin] PATH_INFO: `/v3/users/9ef7004aeef04a0f8376c460bab263d3` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:20.484271 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca0812-c5aa-4a12-9fec-31d79b0e5c0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ef7004aeef04a0f8376c460bab263d3)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:20.489116 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca0812-c5aa-4a12-9fec-31d79b0e5c0e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:20.565702 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-ccca0812-c5aa-4a12-9fec-31d79b0e5c0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ef7004aeef04a0f8376c460bab263d3'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:20.566407 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-ccca0812-c5aa-4a12-9fec-31d79b0e5c0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ef7004aeef04a0f8376c460bab263d3'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:20.589841 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-ccca0812-c5aa-4a12-9fec-31d79b0e5c0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ef7004aeef04a0f8376c460bab263d3'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:20.601697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-ccca0812-c5aa-4a12-9fec-31d79b0e5c0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ef7004aeef04a0f8376c460bab263d3'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:20.622594 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-ccca0812-c5aa-4a12-9fec-31d79b0e5c0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ef7004aeef04a0f8376c460bab263d3'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:20.639460 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1242/2440] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:56:20 2022] DELETE /identity/v3/users/9ef7004aeef04a0f8376c460bab263d3 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:56:20.643857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f510d38-8a9f-4fbb-9ea6-671b05577543 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:20.644330 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f510d38-8a9f-4fbb-9ea6-671b05577543 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:20.644427 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f510d38-8a9f-4fbb-9ea6-671b05577543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:20.664109 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-5f510d38-8a9f-4fbb-9ea6-671b05577543 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:56:20.665727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5f510d38-8a9f-4fbb-9ea6-671b05577543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:20.758684 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1199/2441] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:56:20.779517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca0812-c5aa-4a12-9fec-31d79b0e5c0e admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:20.781782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ccca0812-c5aa-4a12-9fec-31d79b0e5c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:20.855602 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca0812-c5aa-4a12-9fec-31d79b0e5c0e admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:20.863885 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6dc4c6-c6ad-4914-b219-0c104195bb12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:20.864646 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6dc4c6-c6ad-4914-b219-0c104195bb12 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:20.864933 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6dc4c6-c6ad-4914-b219-0c104195bb12 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:20.865212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6dc4c6-c6ad-4914-b219-0c104195bb12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:20.876102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6dc4c6-c6ad-4914-b219-0c104195bb12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:20.877538 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6dc4c6-c6ad-4914-b219-0c104195bb12 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:20.887944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1243/2442] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:56:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:56:21.206245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f510d38-8a9f-4fbb-9ea6-671b05577543 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:21.208590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5f510d38-8a9f-4fbb-9ea6-671b05577543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:21.237773 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f510d38-8a9f-4fbb-9ea6-671b05577543 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:21.246748 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342a363-0809-4c24-a73b-797902ba7ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:21.247795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342a363-0809-4c24-a73b-797902ba7ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:21.248180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342a363-0809-4c24-a73b-797902ba7ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:21.248527 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342a363-0809-4c24-a73b-797902ba7ff6 admin admin] PATH_INFO: `/v3/projects/3dd1eb8097fd42bfa076ac2719a62039` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:21.249892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342a363-0809-4c24-a73b-797902ba7ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dd1eb8097fd42bfa076ac2719a62039)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:21.251861 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342a363-0809-4c24-a73b-797902ba7ff6 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:21.318209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2342a363-0809-4c24-a73b-797902ba7ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dd1eb8097fd42bfa076ac2719a62039', 'request_id': 'req-2342a363-0809-4c24-a73b-797902ba7ff6'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:21.330157 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2342a363-0809-4c24-a73b-797902ba7ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dd1eb8097fd42bfa076ac2719a62039', 'request_id': 'req-2342a363-0809-4c24-a73b-797902ba7ff6'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:21.338133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2342a363-0809-4c24-a73b-797902ba7ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dd1eb8097fd42bfa076ac2719a62039', 'request_id': 'req-2342a363-0809-4c24-a73b-797902ba7ff6'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:21.348295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:56:21.353613 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2342a363-0809-4c24-a73b-797902ba7ff6 admin admin] The token cache is being invalidate because project 3dd1eb8097fd42bfa076ac2719a62039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:56:21.354253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2342a363-0809-4c24-a73b-797902ba7ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:21.355493 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2342a363-0809-4c24-a73b-797902ba7ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:21.357980 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1200/2443] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:56:21 2022] DELETE /identity/v3/projects/3dd1eb8097fd42bfa076ac2719a62039 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:56:23.388024 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6dc4c6-c6ad-4914-b219-0c104195bb12 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:23.389605 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-be6dc4c6-c6ad-4914-b219-0c104195bb12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:23.407299 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6dc4c6-c6ad-4914-b219-0c104195bb12 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:23.505105 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cde1a-64f4-4788-9e5a-58cec6f7f3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:23.506680 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cde1a-64f4-4788-9e5a-58cec6f7f3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:23.506908 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cde1a-64f4-4788-9e5a-58cec6f7f3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:23.507187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cde1a-64f4-4788-9e5a-58cec6f7f3c6 admin admin] PATH_INFO: `/v3/users/a40b5e463cbc4581a728f36235795e47` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:23.508466 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cde1a-64f4-4788-9e5a-58cec6f7f3c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a40b5e463cbc4581a728f36235795e47)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:23.510408 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cde1a-64f4-4788-9e5a-58cec6f7f3c6 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:23.525225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:56:23.562788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-a27cde1a-64f4-4788-9e5a-58cec6f7f3c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a40b5e463cbc4581a728f36235795e47'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:23.563432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-a27cde1a-64f4-4788-9e5a-58cec6f7f3c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a40b5e463cbc4581a728f36235795e47'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:23.583919 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-a27cde1a-64f4-4788-9e5a-58cec6f7f3c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a40b5e463cbc4581a728f36235795e47'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:23.594021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-a27cde1a-64f4-4788-9e5a-58cec6f7f3c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a40b5e463cbc4581a728f36235795e47'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:23.603804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-a27cde1a-64f4-4788-9e5a-58cec6f7f3c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a40b5e463cbc4581a728f36235795e47'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:23.610728 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1244/2444] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:56:23 2022] DELETE /identity/v3/users/a40b5e463cbc4581a728f36235795e47 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:56:23.616049 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97007a05-7054-4ca7-8475-a13709293e27 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:23.616534 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97007a05-7054-4ca7-8475-a13709293e27 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:23.616794 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97007a05-7054-4ca7-8475-a13709293e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:23.638454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-97007a05-7054-4ca7-8475-a13709293e27 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:56:23.639825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-97007a05-7054-4ca7-8475-a13709293e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:23.737297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1201/2445] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:56:23.749070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cde1a-64f4-4788-9e5a-58cec6f7f3c6 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:23.751216 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a27cde1a-64f4-4788-9e5a-58cec6f7f3c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:23.857817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cde1a-64f4-4788-9e5a-58cec6f7f3c6 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:23.862379 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28e998-d4bd-475c-afc7-84e60bf94ec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:23.863041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28e998-d4bd-475c-afc7-84e60bf94ec0 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:23.863319 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28e998-d4bd-475c-afc7-84e60bf94ec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:23.863595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28e998-d4bd-475c-afc7-84e60bf94ec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:23.879289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28e998-d4bd-475c-afc7-84e60bf94ec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:23.880768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28e998-d4bd-475c-afc7-84e60bf94ec0 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:23.899671 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1245/2446] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:56:24.404975 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97007a05-7054-4ca7-8475-a13709293e27 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:56:24.411216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-97007a05-7054-4ca7-8475-a13709293e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:24.435281 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97007a05-7054-4ca7-8475-a13709293e27 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:56:24.448471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca1a8c-2d4b-458b-a748-b2645fb04d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:56:24.449358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca1a8c-2d4b-458b-a748-b2645fb04d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:24.449670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca1a8c-2d4b-458b-a748-b2645fb04d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:24.450084 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca1a8c-2d4b-458b-a748-b2645fb04d21 admin admin] PATH_INFO: `/v3/projects/16a5186ab14a4f4e9f7333cbcb66a07a` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:24.451383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca1a8c-2d4b-458b-a748-b2645fb04d21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16a5186ab14a4f4e9f7333cbcb66a07a)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:56:24.455030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca1a8c-2d4b-458b-a748-b2645fb04d21 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:56:24.591537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-5dca1a8c-2d4b-458b-a748-b2645fb04d21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16a5186ab14a4f4e9f7333cbcb66a07a', 'request_id': 'req-5dca1a8c-2d4b-458b-a748-b2645fb04d21'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:24.602874 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-5dca1a8c-2d4b-458b-a748-b2645fb04d21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16a5186ab14a4f4e9f7333cbcb66a07a', 'request_id': 'req-5dca1a8c-2d4b-458b-a748-b2645fb04d21'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:24.610095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-5dca1a8c-2d4b-458b-a748-b2645fb04d21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16a5186ab14a4f4e9f7333cbcb66a07a', 'request_id': 'req-5dca1a8c-2d4b-458b-a748-b2645fb04d21'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:24.624580 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-5dca1a8c-2d4b-458b-a748-b2645fb04d21 admin admin] The token cache is being invalidate because project 16a5186ab14a4f4e9f7333cbcb66a07a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:56:24.625209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-5dca1a8c-2d4b-458b-a748-b2645fb04d21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:24.632895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-5dca1a8c-2d4b-458b-a748-b2645fb04d21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:56:24.635185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1202/2447] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:56:24 2022] DELETE /identity/v3/projects/16a5186ab14a4f4e9f7333cbcb66a07a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:56:33.888344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=48,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:56:34.620669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:56:55.138839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bb1af-fde0-4307-8dcf-2cafd08a29bd None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:56:55.139436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bb1af-fde0-4307-8dcf-2cafd08a29bd None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:56:55.139436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bb1af-fde0-4307-8dcf-2cafd08a29bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:56:55.157677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-291bb1af-fde0-4307-8dcf-2cafd08a29bd None None] MFA Rules not processed for user `2b3617b8bcac4227b1295e294eec664a`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:56:55.158891 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-291bb1af-fde0-4307-8dcf-2cafd08a29bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:56:55.212921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1246/2448] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:56:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:57:05.206278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=25 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:57:27.518218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca1a8c-2d4b-458b-a748-b2645fb04d21 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:57:27.519627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5dca1a8c-2d4b-458b-a748-b2645fb04d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:57:27.578030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca1a8c-2d4b-458b-a748-b2645fb04d21 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:57:27.581687 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a110a-15c9-4c1c-8415-bb82ea1c565f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:57:27.582340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a110a-15c9-4c1c-8415-bb82ea1c565f service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:57:27.582627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a110a-15c9-4c1c-8415-bb82ea1c565f service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:57:27.582907 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a110a-15c9-4c1c-8415-bb82ea1c565f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:57:27.584728 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b71a110a-15c9-4c1c-8415-bb82ea1c565f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:57:27.665789 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a110a-15c9-4c1c-8415-bb82ea1c565f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:57:27.667156 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a110a-15c9-4c1c-8415-bb82ea1c565f service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:57:27.678571 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1203/2449] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:57:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:57:28.584099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291bb1af-fde0-4307-8dcf-2cafd08a29bd None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:57:28.585524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-291bb1af-fde0-4307-8dcf-2cafd08a29bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:57:28.641301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291bb1af-fde0-4307-8dcf-2cafd08a29bd None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:57:28.644758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95928c7-5a22-4814-8ff8-53b1bd07dd20 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d596de177e24ca5a254bda6644a07da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:57:28.645392 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95928c7-5a22-4814-8ff8-53b1bd07dd20 service glance] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:57:28.645671 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95928c7-5a22-4814-8ff8-53b1bd07dd20 service glance] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:57:28.645941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95928c7-5a22-4814-8ff8-53b1bd07dd20 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:57:28.647852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d95928c7-5a22-4814-8ff8-53b1bd07dd20 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:57:28.662417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95928c7-5a22-4814-8ff8-53b1bd07dd20 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:57:28.663918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95928c7-5a22-4814-8ff8-53b1bd07dd20 service glance] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:57:28.676088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1247/2450] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:57:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:57:29.158949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a110a-15c9-4c1c-8415-bb82ea1c565f service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:57:29.168952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a110a-15c9-4c1c-8415-bb82ea1c565f service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:57:29.172808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b845f52-6075-445a-97c3-88aea141aae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:57:29.173533 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b845f52-6075-445a-97c3-88aea141aae5 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:57:29.173858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b845f52-6075-445a-97c3-88aea141aae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:57:29.174186 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b845f52-6075-445a-97c3-88aea141aae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:57:29.176245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-4b845f52-6075-445a-97c3-88aea141aae5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:57:29.239764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b845f52-6075-445a-97c3-88aea141aae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:57:29.241201 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b845f52-6075-445a-97c3-88aea141aae5 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:57:29.251564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1204/2451] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:57:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:57:38.670765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=26 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:57:39.246449 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=84 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:57:45.318894 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95928c7-5a22-4814-8ff8-53b1bd07dd20 service glance] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:57:45.320334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d95928c7-5a22-4814-8ff8-53b1bd07dd20 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:57:45.334307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95928c7-5a22-4814-8ff8-53b1bd07dd20 service glance] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:57:45.339817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327f142-f05c-4fe6-9a84-f04d03ac52ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:57:45.340453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327f142-f05c-4fe6-9a84-f04d03ac52ff service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:57:45.340738 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327f142-f05c-4fe6-9a84-f04d03ac52ff service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:57:45.341044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327f142-f05c-4fe6-9a84-f04d03ac52ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:57:45.343789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d327f142-f05c-4fe6-9a84-f04d03ac52ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:57:45.402795 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327f142-f05c-4fe6-9a84-f04d03ac52ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:57:45.404958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327f142-f05c-4fe6-9a84-f04d03ac52ff service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:57:45.414197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1248/2452] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:57:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:57:49.925741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b0eb2-c113-49a7-9998-4a783fe11e8d None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:57:49.926258 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b0eb2-c113-49a7-9998-4a783fe11e8d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:57:49.926363 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b0eb2-c113-49a7-9998-4a783fe11e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:57:49.944524 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-6f9b0eb2-c113-49a7-9998-4a783fe11e8d None None] MFA Rules not processed for user `cc44212b992e4358bb37a1f8c4a6a77c`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:57:49.945740 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6f9b0eb2-c113-49a7-9998-4a783fe11e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:57:50.000201 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1205/2453] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:57:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:57:50.538557 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327f142-f05c-4fe6-9a84-f04d03ac52ff service nova] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:57:50.540006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d327f142-f05c-4fe6-9a84-f04d03ac52ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:57:50.612178 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327f142-f05c-4fe6-9a84-f04d03ac52ff service nova] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:57:50.617454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb49f93-db94-4fe9-9aed-8e6c43a61f93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:57:50.618100 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb49f93-db94-4fe9-9aed-8e6c43a61f93 service placement] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:57:50.618381 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb49f93-db94-4fe9-9aed-8e6c43a61f93 service placement] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:57:50.618657 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb49f93-db94-4fe9-9aed-8e6c43a61f93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:57:50.620502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-cdb49f93-db94-4fe9-9aed-8e6c43a61f93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:57:50.632753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb49f93-db94-4fe9-9aed-8e6c43a61f93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:57:50.634369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb49f93-db94-4fe9-9aed-8e6c43a61f93 service placement] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:57:50.644672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1249/2454] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:57:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:57:59.992782 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=25 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:58:00.640526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=60 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:58:01.660609 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3af2f-df38-433d-9cfe-197a9f21ae6d None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:01.661099 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3af2f-df38-433d-9cfe-197a9f21ae6d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:01.661205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3af2f-df38-433d-9cfe-197a9f21ae6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:01.679513 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-86e3af2f-df38-433d-9cfe-197a9f21ae6d None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:58:01.680745 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-86e3af2f-df38-433d-9cfe-197a9f21ae6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:58:01.736820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1206/2455] 10.222.0.60 () {58 vars in 1009 bytes} [Thu Jul 21 20:58:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:58:01.783208 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb49f93-db94-4fe9-9aed-8e6c43a61f93 service placement] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:58:01.784627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-cdb49f93-db94-4fe9-9aed-8e6c43a61f93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:58:01.854385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb49f93-db94-4fe9-9aed-8e6c43a61f93 service placement] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:58:01.860312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf0b8e-275b-4af6-870a-0ea26861530b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:58:01.860960 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf0b8e-275b-4af6-870a-0ea26861530b service swift] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:01.861246 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf0b8e-275b-4af6-870a-0ea26861530b service swift] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:01.861781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf0b8e-275b-4af6-870a-0ea26861530b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:01.870130 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf0b8e-275b-4af6-870a-0ea26861530b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:58:01.871772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf0b8e-275b-4af6-870a-0ea26861530b service swift] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:58:01.880526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1250/2456] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 20:58:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:58:11.729472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=25 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:58:11.877270 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=32 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:58:14.655392 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3af2f-df38-433d-9cfe-197a9f21ae6d None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:58:14.656955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-86e3af2f-df38-433d-9cfe-197a9f21ae6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:58:14.668388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3af2f-df38-433d-9cfe-197a9f21ae6d None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:58:14.672060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0ced7-c511-4064-bc42-e46533556d7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:58:14.672704 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0ced7-c511-4064-bc42-e46533556d7d service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:14.672987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0ced7-c511-4064-bc42-e46533556d7d service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:14.673271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0ced7-c511-4064-bc42-e46533556d7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:14.675007 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-03b0ced7-c511-4064-bc42-e46533556d7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:58:14.684474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0ced7-c511-4064-bc42-e46533556d7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:58:14.686655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0ced7-c511-4064-bc42-e46533556d7d service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:58:14.694603 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1207/2457] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:58:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:58:23.360173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef718ae-1488-4535-a27e-02122e89811a None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:23.360762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef718ae-1488-4535-a27e-02122e89811a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:23.360762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef718ae-1488-4535-a27e-02122e89811a None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:23.362499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1251/2458] 10.222.0.60 () {54 vars in 921 bytes} [Thu Jul 21 20:58:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:58:23.368299 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f6203-a926-4bf4-8fe9-1a54aaff453d None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:23.368756 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f6203-a926-4bf4-8fe9-1a54aaff453d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:23.368868 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f6203-a926-4bf4-8fe9-1a54aaff453d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:23.387193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-440f6203-a926-4bf4-8fe9-1a54aaff453d None None] MFA Rules not processed for user `77812b356c974d2b80102217956169db`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:58:23.388440 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-440f6203-a926-4bf4-8fe9-1a54aaff453d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:58:23.400622 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1208/2459] 10.222.0.60 () {58 vars in 1018 bytes} [Thu Jul 21 20:58:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:58:23.413604 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef718ae-1488-4535-a27e-02122e89811a None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:58:23.430924 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef718ae-1488-4535-a27e-02122e89811a None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:58:23.434967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d83ae9-5749-4154-84b2-f4f48e6152ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d596de177e24ca5a254bda6644a07da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:58:23.436535 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d83ae9-5749-4154-84b2-f4f48e6152ac service glance] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:23.436835 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d83ae9-5749-4154-84b2-f4f48e6152ac service glance] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:23.437174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d83ae9-5749-4154-84b2-f4f48e6152ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:23.448426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d83ae9-5749-4154-84b2-f4f48e6152ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:58:23.449851 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d83ae9-5749-4154-84b2-f4f48e6152ac service glance] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:58:23.461209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1252/2460] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:58:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:58:23.527959 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ccfd0-db6c-4dfa-a352-490cbfb0816f None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:23.528245 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ccfd0-db6c-4dfa-a352-490cbfb0816f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:23.528549 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ccfd0-db6c-4dfa-a352-490cbfb0816f None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:23.529967 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1209/2461] 10.222.0.60 () {54 vars in 921 bytes} [Thu Jul 21 20:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:58:23.534630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb296d-b3c1-476d-83be-6c4d432bebb7 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:23.534949 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb296d-b3c1-476d-83be-6c4d432bebb7 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:23.535229 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb296d-b3c1-476d-83be-6c4d432bebb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:23.555331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-13bb296d-b3c1-476d-83be-6c4d432bebb7 None None] MFA Rules not processed for user `33aff48933844401b86eb808937e35c4`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:58:23.556582 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-13bb296d-b3c1-476d-83be-6c4d432bebb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:58:23.563762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1253/2462] 10.222.0.60 () {58 vars in 1018 bytes} [Thu Jul 21 20:58:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:58:23.575222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ccfd0-db6c-4dfa-a352-490cbfb0816f None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:58:23.586431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ccfd0-db6c-4dfa-a352-490cbfb0816f None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:58:23.590017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e088a-2f39-4c69-be20-c18fe920b8e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:58:23.590655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e088a-2f39-4c69-be20-c18fe920b8e9 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:23.590942 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e088a-2f39-4c69-be20-c18fe920b8e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:23.591222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e088a-2f39-4c69-be20-c18fe920b8e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:23.599110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e088a-2f39-4c69-be20-c18fe920b8e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:58:23.600569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e088a-2f39-4c69-be20-c18fe920b8e9 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:58:23.609380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1210/2463] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:58:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:58:23.693525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e17fea-ba66-4597-85eb-0a200aa0d2e3 None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:23.693837 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e17fea-ba66-4597-85eb-0a200aa0d2e3 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:23.694137 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e17fea-ba66-4597-85eb-0a200aa0d2e3 None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:23.695524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1254/2464] 10.222.0.60 () {54 vars in 921 bytes} [Thu Jul 21 20:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 20:58:23.700479 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4740eb-5cca-4e82-98e6-22c95bd1d780 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:23.700771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4740eb-5cca-4e82-98e6-22c95bd1d780 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:23.701069 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4740eb-5cca-4e82-98e6-22c95bd1d780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:23.719433 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-0e4740eb-5cca-4e82-98e6-22c95bd1d780 None None] MFA Rules not processed for user `77812b356c974d2b80102217956169db`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:58:23.720665 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-0e4740eb-5cca-4e82-98e6-22c95bd1d780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:58:23.730004 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1211/2465] 10.222.0.60 () {58 vars in 1018 bytes} [Thu Jul 21 20:58:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:58:23.741338 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e17fea-ba66-4597-85eb-0a200aa0d2e3 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:58:23.752380 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e17fea-ba66-4597-85eb-0a200aa0d2e3 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:58:23.756021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ef760-f475-4287-9446-6dfa2d5d61ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:58:23.756651 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ef760-f475-4287-9446-6dfa2d5d61ec service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:23.756937 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ef760-f475-4287-9446-6dfa2d5d61ec service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:23.757214 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ef760-f475-4287-9446-6dfa2d5d61ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:23.767370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ef760-f475-4287-9446-6dfa2d5d61ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:58:23.768839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ef760-f475-4287-9446-6dfa2d5d61ec service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:58:23.780211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1255/2466] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:58:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:58:31.194705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bae5bc-dc17-44a2-a0ab-ce8085c4f411 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:31.195254 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bae5bc-dc17-44a2-a0ab-ce8085c4f411 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:31.195625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bae5bc-dc17-44a2-a0ab-ce8085c4f411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:31.215279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d2bae5bc-dc17-44a2-a0ab-ce8085c4f411 None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:58:31.216514 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d2bae5bc-dc17-44a2-a0ab-ce8085c4f411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:58:31.225265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1212/2467] 10.222.0.60 () {58 vars in 1009 bytes} [Thu Jul 21 20:58:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:58:31.238519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ef760-f475-4287-9446-6dfa2d5d61ec service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:58:31.250592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ef760-f475-4287-9446-6dfa2d5d61ec service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:58:31.256478 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c31989-ea32-47a3-bb14-1beace6dbcf9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:58:31.257289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c31989-ea32-47a3-bb14-1beace6dbcf9 service swift] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:31.257641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c31989-ea32-47a3-bb14-1beace6dbcf9 service swift] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:31.257987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c31989-ea32-47a3-bb14-1beace6dbcf9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:31.267035 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c31989-ea32-47a3-bb14-1beace6dbcf9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:58:31.268743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c31989-ea32-47a3-bb14-1beace6dbcf9 service swift] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:58:31.277716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1256/2468] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 20:58:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:58:41.209278 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=21 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:58:41.274100 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=21 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:58:52.334197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bae5bc-dc17-44a2-a0ab-ce8085c4f411 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:58:52.336293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d2bae5bc-dc17-44a2-a0ab-ce8085c4f411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:58:52.352426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bae5bc-dc17-44a2-a0ab-ce8085c4f411 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:58:52.358261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63062798-c007-411d-b414-d60fb571d83f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:58:52.358927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63062798-c007-411d-b414-d60fb571d83f service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:52.359213 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63062798-c007-411d-b414-d60fb571d83f service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:52.359486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63062798-c007-411d-b414-d60fb571d83f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:52.361390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-63062798-c007-411d-b414-d60fb571d83f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:58:52.377767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63062798-c007-411d-b414-d60fb571d83f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:58:52.379630 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63062798-c007-411d-b414-d60fb571d83f service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:58:52.392614 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1213/2469] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:58:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:58:57.962591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c31989-ea32-47a3-bb14-1beace6dbcf9 service swift] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:58:57.977898 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c31989-ea32-47a3-bb14-1beace6dbcf9 service swift] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:58:57.984089 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f9b5a-fc0f-4228-833f-9536a6fcbc20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:58:57.984982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f9b5a-fc0f-4228-833f-9536a6fcbc20 service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:58:57.992955 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f9b5a-fc0f-4228-833f-9536a6fcbc20 service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:58:57.993244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f9b5a-fc0f-4228-833f-9536a6fcbc20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:58:57.995204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e04f9b5a-fc0f-4228-833f-9536a6fcbc20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:58:58.013010 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f9b5a-fc0f-4228-833f-9536a6fcbc20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:58:58.013010 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f9b5a-fc0f-4228-833f-9536a6fcbc20 service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:58:58.024554 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1257/2470] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:58:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 20:59:02.387024 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=6 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:59:04.835536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ff4a3-2dcc-43e0-b34e-87a83b7b09eb None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:59:04.836275 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ff4a3-2dcc-43e0-b34e-87a83b7b09eb None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:59:04.836275 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ff4a3-2dcc-43e0-b34e-87a83b7b09eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:59:04.854367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-608ff4a3-2dcc-43e0-b34e-87a83b7b09eb None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:59:04.855576 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-608ff4a3-2dcc-43e0-b34e-87a83b7b09eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:59:04.863252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1214/2471] 10.222.0.60 () {58 vars in 967 bytes} [Thu Jul 21 20:59:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:59:04.887611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f9b5a-fc0f-4228-833f-9536a6fcbc20 service nova] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:59:04.900635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f9b5a-fc0f-4228-833f-9536a6fcbc20 service nova] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:59:04.905868 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472b8cc-396f-45d3-893a-938048295cff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:59:04.906511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472b8cc-396f-45d3-893a-938048295cff service swift] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:59:04.906798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472b8cc-396f-45d3-893a-938048295cff service swift] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:59:04.907080 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472b8cc-396f-45d3-893a-938048295cff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:59:04.915590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472b8cc-396f-45d3-893a-938048295cff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:59:04.917257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472b8cc-396f-45d3-893a-938048295cff service swift] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:59:04.925497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1258/2472] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 20:59:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 20:59:14.848901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:59:14.922204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=12 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:59:19.401092 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ff4a3-2dcc-43e0-b34e-87a83b7b09eb None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:59:19.414876 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ff4a3-2dcc-43e0-b34e-87a83b7b09eb None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:59:19.420645 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af65729-c3d1-4a4f-829c-4a7f73915c10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:59:19.421303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af65729-c3d1-4a4f-829c-4a7f73915c10 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:59:19.421587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af65729-c3d1-4a4f-829c-4a7f73915c10 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:59:19.421865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af65729-c3d1-4a4f-829c-4a7f73915c10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:59:19.423695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-3af65729-c3d1-4a4f-829c-4a7f73915c10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:59:19.486021 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af65729-c3d1-4a4f-829c-4a7f73915c10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:59:19.487620 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af65729-c3d1-4a4f-829c-4a7f73915c10 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:59:19.497132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1215/2473] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:59:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:59:23.016241 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472b8cc-396f-45d3-893a-938048295cff service swift] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:59:23.017826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-7472b8cc-396f-45d3-893a-938048295cff service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:59:23.032759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472b8cc-396f-45d3-893a-938048295cff service swift] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:59:23.037845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d660fd6-654c-461f-89ac-cf6cd7141f9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:59:23.038490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d660fd6-654c-461f-89ac-cf6cd7141f9b service placement] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:59:23.038781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d660fd6-654c-461f-89ac-cf6cd7141f9b service placement] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:59:23.039096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d660fd6-654c-461f-89ac-cf6cd7141f9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:59:23.041264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6d660fd6-654c-461f-89ac-cf6cd7141f9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:59:23.053159 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d660fd6-654c-461f-89ac-cf6cd7141f9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:59:23.055311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d660fd6-654c-461f-89ac-cf6cd7141f9b service placement] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:59:23.066110 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1259/2474] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 20:59:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 20:59:23.093280 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af65729-c3d1-4a4f-829c-4a7f73915c10 service nova] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:59:23.095561 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-3af65729-c3d1-4a4f-829c-4a7f73915c10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:59:23.163225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af65729-c3d1-4a4f-829c-4a7f73915c10 service nova] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:59:23.168114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd02b0f-0eac-47d5-a4ce-66f32c539ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:59:23.168987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd02b0f-0eac-47d5-a4ce-66f32c539ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:59:23.169270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd02b0f-0eac-47d5-a4ce-66f32c539ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:59:23.169572 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd02b0f-0eac-47d5-a4ce-66f32c539ffd admin admin] PATH_INFO: `/v3/users/cc44212b992e4358bb37a1f8c4a6a77c` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:59:23.170779 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd02b0f-0eac-47d5-a4ce-66f32c539ffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc44212b992e4358bb37a1f8c4a6a77c)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:59:23.172753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd02b0f-0eac-47d5-a4ce-66f32c539ffd admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:59:23.219906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-edd02b0f-0eac-47d5-a4ce-66f32c539ffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc44212b992e4358bb37a1f8c4a6a77c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:59:23.226561 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-edd02b0f-0eac-47d5-a4ce-66f32c539ffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc44212b992e4358bb37a1f8c4a6a77c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:59:23.227195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-edd02b0f-0eac-47d5-a4ce-66f32c539ffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc44212b992e4358bb37a1f8c4a6a77c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:59:23.234180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-edd02b0f-0eac-47d5-a4ce-66f32c539ffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc44212b992e4358bb37a1f8c4a6a77c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:59:23.234811 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-edd02b0f-0eac-47d5-a4ce-66f32c539ffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc44212b992e4358bb37a1f8c4a6a77c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:59:23.252611 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1216/2475] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 20:59:23 2022] DELETE /identity/v3/users/cc44212b992e4358bb37a1f8c4a6a77c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:59:23.434110 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d660fd6-654c-461f-89ac-cf6cd7141f9b service placement] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 20:59:23.445275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d660fd6-654c-461f-89ac-cf6cd7141f9b service placement] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 20:59:23.506326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d9620-5252-4c04-a499-fa60d15e89fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 20:59:23.507193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9620-5252-4c04-a499-fa60d15e89fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:59:23.507477 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9620-5252-4c04-a499-fa60d15e89fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:59:23.507838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9620-5252-4c04-a499-fa60d15e89fb admin admin] PATH_INFO: `/v3/projects/98e1d76644d94fd9ada9ff1ae3267bf8` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:59:23.509072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d9620-5252-4c04-a499-fa60d15e89fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=98e1d76644d94fd9ada9ff1ae3267bf8)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 20:59:23.510963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d9620-5252-4c04-a499-fa60d15e89fb admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 20:59:23.557672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-806d9620-5252-4c04-a499-fa60d15e89fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98e1d76644d94fd9ada9ff1ae3267bf8', 'request_id': 'req-806d9620-5252-4c04-a499-fa60d15e89fb'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:59:23.563753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-806d9620-5252-4c04-a499-fa60d15e89fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98e1d76644d94fd9ada9ff1ae3267bf8', 'request_id': 'req-806d9620-5252-4c04-a499-fa60d15e89fb'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:59:23.569182 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-806d9620-5252-4c04-a499-fa60d15e89fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98e1d76644d94fd9ada9ff1ae3267bf8', 'request_id': 'req-806d9620-5252-4c04-a499-fa60d15e89fb'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:59:23.581478 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-806d9620-5252-4c04-a499-fa60d15e89fb admin admin] The token cache is being invalidate because project 98e1d76644d94fd9ada9ff1ae3267bf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 20:59:23.581768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-806d9620-5252-4c04-a499-fa60d15e89fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:59:23.582412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-806d9620-5252-4c04-a499-fa60d15e89fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 20:59:23.584266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1260/2476] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 20:59:23 2022] DELETE /identity/v3/projects/98e1d76644d94fd9ada9ff1ae3267bf8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 20:59:33.249009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:59:33.578330 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 20:59:49.135209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c93045-7555-4166-86ba-5e6b3dcf8e8d None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 20:59:49.135798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c93045-7555-4166-86ba-5e6b3dcf8e8d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 20:59:49.135798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c93045-7555-4166-86ba-5e6b3dcf8e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 20:59:49.153952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-11c93045-7555-4166-86ba-5e6b3dcf8e8d None None] MFA Rules not processed for user `24e4efb87cd34008a1b83c0ea0cce5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 20:59:49.155161 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-11c93045-7555-4166-86ba-5e6b3dcf8e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 20:59:49.209974 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1217/2477] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 20:59:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 20:59:59.203223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=25 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:00:23.218436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1ce61-4229-4d55-bdc4-21a94d1c3057 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:00:23.218982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1ce61-4229-4d55-bdc4-21a94d1c3057 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:00:23.218982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1ce61-4229-4d55-bdc4-21a94d1c3057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:00:23.246097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-f6d1ce61-4229-4d55-bdc4-21a94d1c3057 None None] MFA Rules not processed for user `3a0d900f08d54098870ebbfa281e2a65`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:00:23.247309 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f6d1ce61-4229-4d55-bdc4-21a94d1c3057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:00:23.318222 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1261/2478] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:00:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:00:23.620271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c93045-7555-4166-86ba-5e6b3dcf8e8d None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:00:23.621712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-11c93045-7555-4166-86ba-5e6b3dcf8e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:00:23.708312 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c93045-7555-4166-86ba-5e6b3dcf8e8d None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:00:23.714325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1714692-00c2-49ac-9592-ec4b8b37def4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:00:23.714991 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1714692-00c2-49ac-9592-ec4b8b37def4 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:00:23.715277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1714692-00c2-49ac-9592-ec4b8b37def4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:00:23.715555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1714692-00c2-49ac-9592-ec4b8b37def4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:00:23.719004 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e1714692-00c2-49ac-9592-ec4b8b37def4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:00:23.730839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1714692-00c2-49ac-9592-ec4b8b37def4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:00:23.732412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1714692-00c2-49ac-9592-ec4b8b37def4 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:00:23.740632 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1218/2479] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:00:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:00:27.495206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1ce61-4229-4d55-bdc4-21a94d1c3057 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:00:27.496628 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f6d1ce61-4229-4d55-bdc4-21a94d1c3057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:00:27.611164 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1ce61-4229-4d55-bdc4-21a94d1c3057 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:00:27.617192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef3b40-d0e6-4bc9-a5b7-7c99f5a90245 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:00:27.617846 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef3b40-d0e6-4bc9-a5b7-7c99f5a90245 service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:00:27.618129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef3b40-d0e6-4bc9-a5b7-7c99f5a90245 service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:00:27.618402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef3b40-d0e6-4bc9-a5b7-7c99f5a90245 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:00:27.620423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c2ef3b40-d0e6-4bc9-a5b7-7c99f5a90245 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:00:27.667346 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:00:27.683491 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef3b40-d0e6-4bc9-a5b7-7c99f5a90245 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:00:27.685102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef3b40-d0e6-4bc9-a5b7-7c99f5a90245 service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:00:27.694853 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1262/2480] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:00:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:00:33.736847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=39 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:00:37.690678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=8 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:00:55.960134 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1714692-00c2-49ac-9592-ec4b8b37def4 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:00:55.961663 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e1714692-00c2-49ac-9592-ec4b8b37def4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:00:56.068460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1714692-00c2-49ac-9592-ec4b8b37def4 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:00:56.073702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60bf8a-299f-49f6-be2d-4cb42209de1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:00:56.074364 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60bf8a-299f-49f6-be2d-4cb42209de1e service placement] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:00:56.074645 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60bf8a-299f-49f6-be2d-4cb42209de1e service placement] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:00:56.074923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60bf8a-299f-49f6-be2d-4cb42209de1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:00:56.077017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-3b60bf8a-299f-49f6-be2d-4cb42209de1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:00:56.089282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60bf8a-299f-49f6-be2d-4cb42209de1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:00:56.090898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60bf8a-299f-49f6-be2d-4cb42209de1e service placement] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:00:56.105428 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1219/2481] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:00:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:00:57.848418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ae74a-09ea-438f-8b08-db2c59f90e89 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:00:57.848897 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ae74a-09ea-438f-8b08-db2c59f90e89 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:00:57.848998 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ae74a-09ea-438f-8b08-db2c59f90e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:00:57.866920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e64ae74a-09ea-438f-8b08-db2c59f90e89 None None] MFA Rules not processed for user `2b3617b8bcac4227b1295e294eec664a`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:00:57.868190 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e64ae74a-09ea-438f-8b08-db2c59f90e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:00:57.923431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1263/2482] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:00:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:00:58.237357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60bf8a-299f-49f6-be2d-4cb42209de1e service placement] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:00:58.238771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-3b60bf8a-299f-49f6-be2d-4cb42209de1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:00:58.271208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60bf8a-299f-49f6-be2d-4cb42209de1e service placement] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:00:58.274809 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76f417-6baf-4dbf-aa25-6976945cf7c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:00:58.275445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76f417-6baf-4dbf-aa25-6976945cf7c1 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:00:58.275755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76f417-6baf-4dbf-aa25-6976945cf7c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:00:58.276041 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76f417-6baf-4dbf-aa25-6976945cf7c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:00:58.277771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7c76f417-6baf-4dbf-aa25-6976945cf7c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:00:58.287629 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76f417-6baf-4dbf-aa25-6976945cf7c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:00:58.289082 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76f417-6baf-4dbf-aa25-6976945cf7c1 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:00:58.297545 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1220/2483] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:00:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:07.916181 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=25 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:08.293436 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=56 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:20.388464 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ae74a-09ea-438f-8b08-db2c59f90e89 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:20.402194 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ae74a-09ea-438f-8b08-db2c59f90e89 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:20.408411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3af780-b7b3-4f89-a852-7c38b0f02cb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:20.409059 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3af780-b7b3-4f89-a852-7c38b0f02cb9 service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:20.409346 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3af780-b7b3-4f89-a852-7c38b0f02cb9 service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:20.409625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3af780-b7b3-4f89-a852-7c38b0f02cb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:20.411422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ca3af780-b7b3-4f89-a852-7c38b0f02cb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:20.422304 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3af780-b7b3-4f89-a852-7c38b0f02cb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:20.424118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3af780-b7b3-4f89-a852-7c38b0f02cb9 service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:20.433265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1264/2484] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:24.748869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76f417-6baf-4dbf-aa25-6976945cf7c1 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:24.763919 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76f417-6baf-4dbf-aa25-6976945cf7c1 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:24.770541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a41683-9619-4add-b9f3-11c503ef4efd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:24.771210 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a41683-9619-4add-b9f3-11c503ef4efd service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:24.771501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a41683-9619-4add-b9f3-11c503ef4efd service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:24.771812 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a41683-9619-4add-b9f3-11c503ef4efd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:24.773636 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-39a41683-9619-4add-b9f3-11c503ef4efd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:24.787770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a41683-9619-4add-b9f3-11c503ef4efd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:24.789624 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a41683-9619-4add-b9f3-11c503ef4efd service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:24.811004 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1221/2485] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:28.308985 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3af780-b7b3-4f89-a852-7c38b0f02cb9 service nova] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:28.318974 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3af780-b7b3-4f89-a852-7c38b0f02cb9 service nova] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:28.322586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47280e02-afbe-4d06-a944-730eac2759b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:28.323252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47280e02-afbe-4d06-a944-730eac2759b5 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:28.323537 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47280e02-afbe-4d06-a944-730eac2759b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:28.323854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47280e02-afbe-4d06-a944-730eac2759b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:28.325618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-47280e02-afbe-4d06-a944-730eac2759b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:28.412178 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47280e02-afbe-4d06-a944-730eac2759b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:28.413638 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47280e02-afbe-4d06-a944-730eac2759b5 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:28.433486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1265/2486] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:31.492925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a41683-9619-4add-b9f3-11c503ef4efd service nova] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:31.494521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-39a41683-9619-4add-b9f3-11c503ef4efd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:31.509225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a41683-9619-4add-b9f3-11c503ef4efd service nova] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:31.514372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d68f2e-8fd2-4295-918f-139246176483 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:31.515019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d68f2e-8fd2-4295-918f-139246176483 service placement] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:31.515308 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d68f2e-8fd2-4295-918f-139246176483 service placement] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:31.515675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d68f2e-8fd2-4295-918f-139246176483 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:31.517513 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e7d68f2e-8fd2-4295-918f-139246176483 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:31.529226 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d68f2e-8fd2-4295-918f-139246176483 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:31.530869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d68f2e-8fd2-4295-918f-139246176483 service placement] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:31.542743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1222/2487] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 21:01:31.573159 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47280e02-afbe-4d06-a944-730eac2759b5 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:31.576340 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-47280e02-afbe-4d06-a944-730eac2759b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:31.589179 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47280e02-afbe-4d06-a944-730eac2759b5 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:31.594875 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583602ff-0650-4ae2-bd2f-407871a18d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:31.595906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583602ff-0650-4ae2-bd2f-407871a18d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:31.596189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583602ff-0650-4ae2-bd2f-407871a18d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:31.596466 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583602ff-0650-4ae2-bd2f-407871a18d28 admin admin] PATH_INFO: `/v3/users/24e4efb87cd34008a1b83c0ea0cce5aa` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:31.597698 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583602ff-0650-4ae2-bd2f-407871a18d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24e4efb87cd34008a1b83c0ea0cce5aa)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:31.599643 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583602ff-0650-4ae2-bd2f-407871a18d28 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:31.651625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-583602ff-0650-4ae2-bd2f-407871a18d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24e4efb87cd34008a1b83c0ea0cce5aa'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:31.652671 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-583602ff-0650-4ae2-bd2f-407871a18d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24e4efb87cd34008a1b83c0ea0cce5aa'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:31.665868 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-583602ff-0650-4ae2-bd2f-407871a18d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24e4efb87cd34008a1b83c0ea0cce5aa'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:31.674355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-583602ff-0650-4ae2-bd2f-407871a18d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24e4efb87cd34008a1b83c0ea0cce5aa'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:31.681422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-583602ff-0650-4ae2-bd2f-407871a18d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24e4efb87cd34008a1b83c0ea0cce5aa'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:31.688222 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1266/2488] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:01:31 2022] DELETE /identity/v3/users/24e4efb87cd34008a1b83c0ea0cce5aa => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:31.867563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d68f2e-8fd2-4295-918f-139246176483 service placement] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:31.881793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d68f2e-8fd2-4295-918f-139246176483 service placement] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:31.941557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97571ccf-c3f1-4deb-aaa5-82f61cce20cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:31.942488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97571ccf-c3f1-4deb-aaa5-82f61cce20cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:31.942768 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97571ccf-c3f1-4deb-aaa5-82f61cce20cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:31.943039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97571ccf-c3f1-4deb-aaa5-82f61cce20cc admin admin] PATH_INFO: `/v3/projects/d9e9d8f8b55c4ef586252456ea195266` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:31.944254 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97571ccf-c3f1-4deb-aaa5-82f61cce20cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e9d8f8b55c4ef586252456ea195266)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:31.946175 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97571ccf-c3f1-4deb-aaa5-82f61cce20cc admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:31.993272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-97571ccf-c3f1-4deb-aaa5-82f61cce20cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e9d8f8b55c4ef586252456ea195266', 'request_id': 'req-97571ccf-c3f1-4deb-aaa5-82f61cce20cc'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:32.000782 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-97571ccf-c3f1-4deb-aaa5-82f61cce20cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e9d8f8b55c4ef586252456ea195266', 'request_id': 'req-97571ccf-c3f1-4deb-aaa5-82f61cce20cc'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:32.006351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-97571ccf-c3f1-4deb-aaa5-82f61cce20cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e9d8f8b55c4ef586252456ea195266', 'request_id': 'req-97571ccf-c3f1-4deb-aaa5-82f61cce20cc'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:32.019338 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-97571ccf-c3f1-4deb-aaa5-82f61cce20cc admin admin] The token cache is being invalidate because project d9e9d8f8b55c4ef586252456ea195266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:01:32.019626 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-97571ccf-c3f1-4deb-aaa5-82f61cce20cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:32.020323 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-97571ccf-c3f1-4deb-aaa5-82f61cce20cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:32.021937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1223/2489] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:01:31 2022] DELETE /identity/v3/projects/d9e9d8f8b55c4ef586252456ea195266 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:01:32.028719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0286f-b2ef-493f-b378-6190b9204093 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:32.029050 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0286f-b2ef-493f-b378-6190b9204093 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:32.029545 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0286f-b2ef-493f-b378-6190b9204093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:32.072205 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:32.101034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-c3c0286f-b2ef-493f-b378-6190b9204093 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:32.102245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c3c0286f-b2ef-493f-b378-6190b9204093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:32.160954 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1267/2490] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:32.168360 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479e05f-85ba-4c5c-b77e-a7407f591070 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:32.168653 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479e05f-85ba-4c5c-b77e-a7407f591070 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:32.168933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479e05f-85ba-4c5c-b77e-a7407f591070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:32.186451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-6479e05f-85ba-4c5c-b77e-a7407f591070 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:32.187654 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6479e05f-85ba-4c5c-b77e-a7407f591070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:32.196842 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1224/2491] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:32.200811 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0286f-b2ef-493f-b378-6190b9204093 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:32.211077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0286f-b2ef-493f-b378-6190b9204093 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:32.215802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fa9d7-4782-4d23-936d-11b9180c4f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:32.216450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fa9d7-4782-4d23-936d-11b9180c4f94 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:32.216734 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fa9d7-4782-4d23-936d-11b9180c4f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:32.217013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fa9d7-4782-4d23-936d-11b9180c4f94 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:32.217737 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266fa9d7-4782-4d23-936d-11b9180c4f94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:01:32.218042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266fa9d7-4782-4d23-936d-11b9180c4f94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:32.219761 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266fa9d7-4782-4d23-936d-11b9180c4f94 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:32.231726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1268/2492] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 21:01:32.236604 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479e05f-85ba-4c5c-b77e-a7407f591070 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:32.246318 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479e05f-85ba-4c5c-b77e-a7407f591070 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:32.251798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aefe7a-50d7-4daa-a5a6-dcd16821005d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:32.252453 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aefe7a-50d7-4daa-a5a6-dcd16821005d admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:32.252741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aefe7a-50d7-4daa-a5a6-dcd16821005d admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:32.253024 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aefe7a-50d7-4daa-a5a6-dcd16821005d admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:32.253726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aefe7a-50d7-4daa-a5a6-dcd16821005d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:32.255648 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aefe7a-50d7-4daa-a5a6-dcd16821005d admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:32.283364 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1225/2493] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:01:32 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:01:32.287497 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fa9d7-4782-4d23-936d-11b9180c4f94 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:32.298402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fa9d7-4782-4d23-936d-11b9180c4f94 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:32.354849 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e030bd3-b253-4516-aa38-c4735899c56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:32.355517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e030bd3-b253-4516-aa38-c4735899c56b admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:32.355836 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e030bd3-b253-4516-aa38-c4735899c56b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:32.356114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e030bd3-b253-4516-aa38-c4735899c56b admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:32.356808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e030bd3-b253-4516-aa38-c4735899c56b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:32.358763 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e030bd3-b253-4516-aa38-c4735899c56b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:32.379513 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1269/2494] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:01:32 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 21:01:32.383646 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aefe7a-50d7-4daa-a5a6-dcd16821005d admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:32.394015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aefe7a-50d7-4daa-a5a6-dcd16821005d admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:32.399941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87775529-702f-4236-97cf-04d5889c4771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:32.400615 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87775529-702f-4236-97cf-04d5889c4771 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:32.400901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87775529-702f-4236-97cf-04d5889c4771 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:32.401236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87775529-702f-4236-97cf-04d5889c4771 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:32.402001 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87775529-702f-4236-97cf-04d5889c4771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:32.403637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87775529-702f-4236-97cf-04d5889c4771 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:32.417160 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1226/2495] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:32.421079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e030bd3-b253-4516-aa38-c4735899c56b admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:32.431663 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e030bd3-b253-4516-aa38-c4735899c56b admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:32.436450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e219c05-61ee-4192-8fd7-745a5661dfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:32.437108 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e219c05-61ee-4192-8fd7-745a5661dfdf admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:32.438032 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e219c05-61ee-4192-8fd7-745a5661dfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:32.438316 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e219c05-61ee-4192-8fd7-745a5661dfdf admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:32.439042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e219c05-61ee-4192-8fd7-745a5661dfdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:32.440667 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e219c05-61ee-4192-8fd7-745a5661dfdf admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:32.450505 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1270/2496] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:32.454737 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87775529-702f-4236-97cf-04d5889c4771 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:32.464740 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87775529-702f-4236-97cf-04d5889c4771 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:32.469907 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3c581-b652-4365-b294-5246a326c422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:32.470769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3c581-b652-4365-b294-5246a326c422 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:32.471047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3c581-b652-4365-b294-5246a326c422 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:32.471322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3c581-b652-4365-b294-5246a326c422 admin admin] PATH_INFO: `/v3/projects/892df247b3b24e4389680218d3dbe7b0/users/953f0d2d0d55496e8ec96375fd88d968/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:32.486567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3c581-b652-4365-b294-5246a326c422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=892df247b3b24e4389680218d3dbe7b0, user_id=953f0d2d0d55496e8ec96375fd88d968, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:32.489472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3c581-b652-4365-b294-5246a326c422 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:32.500134 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:32.513928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1227/2497] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:01:32 2022] PUT /identity/v3/projects/892df247b3b24e4389680218d3dbe7b0/users/953f0d2d0d55496e8ec96375fd88d968/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:01:32.523960 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e324a-154f-4c98-95ce-5a0a3ec85a18 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:32.524253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e324a-154f-4c98-95ce-5a0a3ec85a18 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:32.524536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e324a-154f-4c98-95ce-5a0a3ec85a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:32.553376 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d09e324a-154f-4c98-95ce-5a0a3ec85a18 None None] MFA Rules not processed for user `953f0d2d0d55496e8ec96375fd88d968`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:32.554614 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d09e324a-154f-4c98-95ce-5a0a3ec85a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:32.603302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:32.623060 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1271/2498] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:32.627732 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3c581-b652-4365-b294-5246a326c422 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:32.690292 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3c581-b652-4365-b294-5246a326c422 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:32.695153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae95d20-2b25-4b65-9332-f222d5f21d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:32.695842 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae95d20-2b25-4b65-9332-f222d5f21d19 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:32.696130 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae95d20-2b25-4b65-9332-f222d5f21d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:32.696410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae95d20-2b25-4b65-9332-f222d5f21d19 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:32.697098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae95d20-2b25-4b65-9332-f222d5f21d19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:32.699028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae95d20-2b25-4b65-9332-f222d5f21d19 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:32.725420 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1228/2499] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:01:32 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:01:32.728818 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e324a-154f-4c98-95ce-5a0a3ec85a18 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:32.740106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e324a-154f-4c98-95ce-5a0a3ec85a18 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:32.794498 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808d07d-c884-4ae1-bcbc-50f947a38353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:32.795158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808d07d-c884-4ae1-bcbc-50f947a38353 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:32.795441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808d07d-c884-4ae1-bcbc-50f947a38353 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:32.795730 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808d07d-c884-4ae1-bcbc-50f947a38353 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:32.796443 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808d07d-c884-4ae1-bcbc-50f947a38353 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:32.798608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808d07d-c884-4ae1-bcbc-50f947a38353 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:32.821574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1272/2500] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:01:32 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:01:32.822772 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae95d20-2b25-4b65-9332-f222d5f21d19 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:32.833728 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae95d20-2b25-4b65-9332-f222d5f21d19 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:32.838303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffe7a4-099b-4d9f-8b77-f8c2da899573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:32.838958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffe7a4-099b-4d9f-8b77-f8c2da899573 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:32.839241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffe7a4-099b-4d9f-8b77-f8c2da899573 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:32.839516 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffe7a4-099b-4d9f-8b77-f8c2da899573 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:32.840302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffe7a4-099b-4d9f-8b77-f8c2da899573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:32.841899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffe7a4-099b-4d9f-8b77-f8c2da899573 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:32.854557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1229/2501] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 21:01:32.858507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808d07d-c884-4ae1-bcbc-50f947a38353 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:32.868670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808d07d-c884-4ae1-bcbc-50f947a38353 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:32.873649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf3e64-9a9f-423a-8e5b-a9c62a7c0a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:32.874590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf3e64-9a9f-423a-8e5b-a9c62a7c0a08 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:32.875004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf3e64-9a9f-423a-8e5b-a9c62a7c0a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:32.875288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf3e64-9a9f-423a-8e5b-a9c62a7c0a08 admin admin] PATH_INFO: `/v3/projects/7a29d6a6864f4c2f965568170a346181/users/f87304ad8e49495a8cff6bb8f9715dc1/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:32.892425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf3e64-9a9f-423a-8e5b-a9c62a7c0a08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a29d6a6864f4c2f965568170a346181, user_id=f87304ad8e49495a8cff6bb8f9715dc1, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:32.895098 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf3e64-9a9f-423a-8e5b-a9c62a7c0a08 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:32.908859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1273/2502] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:01:32 2022] PUT /identity/v3/projects/7a29d6a6864f4c2f965568170a346181/users/f87304ad8e49495a8cff6bb8f9715dc1/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:32.912842 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffe7a4-099b-4d9f-8b77-f8c2da899573 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:32.922796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffe7a4-099b-4d9f-8b77-f8c2da899573 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:32.980060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e174f-cc7a-4fc3-8d40-01bae5e9d134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:32.980655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e174f-cc7a-4fc3-8d40-01bae5e9d134 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:32.980939 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e174f-cc7a-4fc3-8d40-01bae5e9d134 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:32.981221 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e174f-cc7a-4fc3-8d40-01bae5e9d134 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:32.981988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e174f-cc7a-4fc3-8d40-01bae5e9d134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:32.983581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e174f-cc7a-4fc3-8d40-01bae5e9d134 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:32.994054 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1230/2503] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:32.998250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf3e64-9a9f-423a-8e5b-a9c62a7c0a08 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:33.008471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf3e64-9a9f-423a-8e5b-a9c62a7c0a08 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:33.013044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df914ac-a3a4-48f6-9b4b-be0b85e5df9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:33.013943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df914ac-a3a4-48f6-9b4b-be0b85e5df9d admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:33.014230 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df914ac-a3a4-48f6-9b4b-be0b85e5df9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:33.014509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df914ac-a3a4-48f6-9b4b-be0b85e5df9d admin admin] PATH_INFO: `/v3/projects/7a29d6a6864f4c2f965568170a346181/users/f87304ad8e49495a8cff6bb8f9715dc1/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:33.017225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df914ac-a3a4-48f6-9b4b-be0b85e5df9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a29d6a6864f4c2f965568170a346181, user_id=f87304ad8e49495a8cff6bb8f9715dc1, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:33.019915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df914ac-a3a4-48f6-9b4b-be0b85e5df9d admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:33.030093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1274/2504] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:01:32 2022] PUT /identity/v3/projects/7a29d6a6864f4c2f965568170a346181/users/f87304ad8e49495a8cff6bb8f9715dc1/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:01:33.041954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24313356-0dc4-431a-8003-1c7793672862 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:33.042464 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24313356-0dc4-431a-8003-1c7793672862 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:33.042755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24313356-0dc4-431a-8003-1c7793672862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:33.068498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-24313356-0dc4-431a-8003-1c7793672862 None None] MFA Rules not processed for user `f87304ad8e49495a8cff6bb8f9715dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:33.069699 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-24313356-0dc4-431a-8003-1c7793672862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:33.130260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:33.140385 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1231/2505] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:33.153247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df914ac-a3a4-48f6-9b4b-be0b85e5df9d admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:33.154955 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-4df914ac-a3a4-48f6-9b4b-be0b85e5df9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:33.234296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df914ac-a3a4-48f6-9b4b-be0b85e5df9d admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:33.238629 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65c8ff-274e-4125-aedc-a5bf6aa1c1f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:33.239308 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65c8ff-274e-4125-aedc-a5bf6aa1c1f9 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:33.239645 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65c8ff-274e-4125-aedc-a5bf6aa1c1f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:33.240208 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65c8ff-274e-4125-aedc-a5bf6aa1c1f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:33.249639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65c8ff-274e-4125-aedc-a5bf6aa1c1f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:33.251933 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65c8ff-274e-4125-aedc-a5bf6aa1c1f9 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:33.261540 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1275/2506] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:33 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:33.557764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24313356-0dc4-431a-8003-1c7793672862 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:33.567791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24313356-0dc4-431a-8003-1c7793672862 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:33.572407 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ea113-0685-488b-9786-c1ab4f7bbdb9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87304ad8e49495a8cff6bb8f9715dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a29d6a6864f4c2f965568170a346181', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:33.573066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ea113-0685-488b-9786-c1ab4f7bbdb9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:33.573356 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ea113-0685-488b-9786-c1ab4f7bbdb9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:33.573637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ea113-0685-488b-9786-c1ab4f7bbdb9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:33.574392 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ea113-0685-488b-9786-c1ab4f7bbdb9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:01:33.574704 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ea113-0685-488b-9786-c1ab4f7bbdb9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:33.576428 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ea113-0685-488b-9786-c1ab4f7bbdb9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:33.588208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1232/2507] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 21:01:33.592247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65c8ff-274e-4125-aedc-a5bf6aa1c1f9 service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:33.601635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65c8ff-274e-4125-aedc-a5bf6aa1c1f9 service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:33.606136 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9876c5-a172-4031-a85c-55b39c0885ff tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87304ad8e49495a8cff6bb8f9715dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a29d6a6864f4c2f965568170a346181', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:33.606774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9876c5-a172-4031-a85c-55b39c0885ff tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:33.607056 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9876c5-a172-4031-a85c-55b39c0885ff tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:33.607339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9876c5-a172-4031-a85c-55b39c0885ff tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:33.608073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9876c5-a172-4031-a85c-55b39c0885ff tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:33.610669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9876c5-a172-4031-a85c-55b39c0885ff tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:33.636603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1276/2508] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:01:33 2022] POST /identity/v3/projects => generated 355 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:01:33.640891 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ea113-0685-488b-9786-c1ab4f7bbdb9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:33.650778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ea113-0685-488b-9786-c1ab4f7bbdb9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:33.706603 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7ec26-d89a-4a23-848d-1cc03c0380b9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87304ad8e49495a8cff6bb8f9715dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a29d6a6864f4c2f965568170a346181', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:33.707269 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ec26-d89a-4a23-848d-1cc03c0380b9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:33.707562 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ec26-d89a-4a23-848d-1cc03c0380b9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:33.707879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ec26-d89a-4a23-848d-1cc03c0380b9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:33.708597 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7ec26-d89a-4a23-848d-1cc03c0380b9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:01:33.708905 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7ec26-d89a-4a23-848d-1cc03c0380b9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:33.710492 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7ec26-d89a-4a23-848d-1cc03c0380b9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:33.722377 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1233/2509] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:01:33.758649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9876c5-a172-4031-a85c-55b39c0885ff tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:33.764325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:33.770839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9876c5-a172-4031-a85c-55b39c0885ff tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:33.775683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03596c94-348c-4177-a602-0ed11940edb0 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87304ad8e49495a8cff6bb8f9715dc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a29d6a6864f4c2f965568170a346181', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:33.776645 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03596c94-348c-4177-a602-0ed11940edb0 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:33.776936 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03596c94-348c-4177-a602-0ed11940edb0 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:33.777217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03596c94-348c-4177-a602-0ed11940edb0 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] PATH_INFO: `/v3/projects/427227a380bb4070b33908078489d4df` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:33.778423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03596c94-348c-4177-a602-0ed11940edb0 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] RBAC: Authorizing `identity:delete_project(project_id=427227a380bb4070b33908078489d4df)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:33.780381 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03596c94-348c-4177-a602-0ed11940edb0 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:33.827049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-03596c94-348c-4177-a602-0ed11940edb0 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '427227a380bb4070b33908078489d4df', 'request_id': 'req-03596c94-348c-4177-a602-0ed11940edb0'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:33.834265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-03596c94-348c-4177-a602-0ed11940edb0 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '427227a380bb4070b33908078489d4df', 'request_id': 'req-03596c94-348c-4177-a602-0ed11940edb0'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:33.839911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-03596c94-348c-4177-a602-0ed11940edb0 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '427227a380bb4070b33908078489d4df', 'request_id': 'req-03596c94-348c-4177-a602-0ed11940edb0'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:33.853063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-03596c94-348c-4177-a602-0ed11940edb0 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] The token cache is being invalidate because project 427227a380bb4070b33908078489d4df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:01:33.853352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-03596c94-348c-4177-a602-0ed11940edb0 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:33.853992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-03596c94-348c-4177-a602-0ed11940edb0 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:33.855544 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1277/2510] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:01:33 2022] DELETE /identity/v3/projects/427227a380bb4070b33908078489d4df => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:34.040799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7ec26-d89a-4a23-848d-1cc03c0380b9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:34.042532 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d0c7ec26-d89a-4a23-848d-1cc03c0380b9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:34.110108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7ec26-d89a-4a23-848d-1cc03c0380b9 tempest-VolumeQuotaClassesTest-350547733 tempest-VolumeQuotaClassesTest-350547733-project-admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:34.167237 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812ae7a-e5c9-4bef-b9a3-a86e4f9544b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:34.168140 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812ae7a-e5c9-4bef-b9a3-a86e4f9544b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:34.168424 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812ae7a-e5c9-4bef-b9a3-a86e4f9544b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:34.168699 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812ae7a-e5c9-4bef-b9a3-a86e4f9544b1 admin admin] PATH_INFO: `/v3/users/953f0d2d0d55496e8ec96375fd88d968` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:34.169902 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812ae7a-e5c9-4bef-b9a3-a86e4f9544b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=953f0d2d0d55496e8ec96375fd88d968)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:34.172443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812ae7a-e5c9-4bef-b9a3-a86e4f9544b1 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:34.200034 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:34.219510 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a812ae7a-e5c9-4bef-b9a3-a86e4f9544b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '953f0d2d0d55496e8ec96375fd88d968'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:34.226045 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a812ae7a-e5c9-4bef-b9a3-a86e4f9544b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '953f0d2d0d55496e8ec96375fd88d968'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:34.226822 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a812ae7a-e5c9-4bef-b9a3-a86e4f9544b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '953f0d2d0d55496e8ec96375fd88d968'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:34.233762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a812ae7a-e5c9-4bef-b9a3-a86e4f9544b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '953f0d2d0d55496e8ec96375fd88d968'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:34.235010 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-a812ae7a-e5c9-4bef-b9a3-a86e4f9544b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '953f0d2d0d55496e8ec96375fd88d968'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:34.253954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1234/2511] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:01:34 2022] DELETE /identity/v3/users/953f0d2d0d55496e8ec96375fd88d968 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:01:34.258557 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b457c5c-0845-477f-aa12-306c20e6b125 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:34.258832 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b457c5c-0845-477f-aa12-306c20e6b125 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:34.259113 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b457c5c-0845-477f-aa12-306c20e6b125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:34.277548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-2b457c5c-0845-477f-aa12-306c20e6b125 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:34.278704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-2b457c5c-0845-477f-aa12-306c20e6b125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:34.340301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1278/2512] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:34.351407 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812ae7a-e5c9-4bef-b9a3-a86e4f9544b1 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:34.352917 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a812ae7a-e5c9-4bef-b9a3-a86e4f9544b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:34.410968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812ae7a-e5c9-4bef-b9a3-a86e4f9544b1 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:34.414780 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5c702-be88-4f3e-a84f-3f67e398f42e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:34.415425 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5c702-be88-4f3e-a84f-3f67e398f42e service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:34.415731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5c702-be88-4f3e-a84f-3f67e398f42e service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:34.416017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5c702-be88-4f3e-a84f-3f67e398f42e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:34.426020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5c702-be88-4f3e-a84f-3f67e398f42e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:34.428359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5c702-be88-4f3e-a84f-3f67e398f42e service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:34.439012 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1235/2513] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:34.706472 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b457c5c-0845-477f-aa12-306c20e6b125 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:34.707990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-2b457c5c-0845-477f-aa12-306c20e6b125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:34.720630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b457c5c-0845-477f-aa12-306c20e6b125 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:34.725254 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:34.726107 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:34.726384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:34.726711 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a admin admin] PATH_INFO: `/v3/projects/892df247b3b24e4389680218d3dbe7b0` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:34.727850 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=892df247b3b24e4389680218d3dbe7b0)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:34.729756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:34.778119 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '892df247b3b24e4389680218d3dbe7b0', 'request_id': 'req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:34.784311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '892df247b3b24e4389680218d3dbe7b0', 'request_id': 'req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:34.789795 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '892df247b3b24e4389680218d3dbe7b0', 'request_id': 'req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:34.801884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a admin admin] The token cache is being invalidate because project 892df247b3b24e4389680218d3dbe7b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:01:34.802171 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:34.802798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:34.804378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1279/2514] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:01:34 2022] DELETE /identity/v3/projects/892df247b3b24e4389680218d3dbe7b0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:34.808507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5c702-be88-4f3e-a84f-3f67e398f42e service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:34.819486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5c702-be88-4f3e-a84f-3f67e398f42e service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:34.879801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308d438-3ab9-4db0-94a5-6bd599af315e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:34.880612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308d438-3ab9-4db0-94a5-6bd599af315e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:34.880893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308d438-3ab9-4db0-94a5-6bd599af315e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:34.881167 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308d438-3ab9-4db0-94a5-6bd599af315e admin admin] PATH_INFO: `/v3/users/f87304ad8e49495a8cff6bb8f9715dc1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:34.882356 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308d438-3ab9-4db0-94a5-6bd599af315e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f87304ad8e49495a8cff6bb8f9715dc1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:34.884383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308d438-3ab9-4db0-94a5-6bd599af315e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:34.930807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2308d438-3ab9-4db0-94a5-6bd599af315e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f87304ad8e49495a8cff6bb8f9715dc1'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:34.932442 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:34.938762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2308d438-3ab9-4db0-94a5-6bd599af315e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f87304ad8e49495a8cff6bb8f9715dc1'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:34.939905 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2308d438-3ab9-4db0-94a5-6bd599af315e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f87304ad8e49495a8cff6bb8f9715dc1'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:34.947894 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2308d438-3ab9-4db0-94a5-6bd599af315e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f87304ad8e49495a8cff6bb8f9715dc1'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:34.948540 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-2308d438-3ab9-4db0-94a5-6bd599af315e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f87304ad8e49495a8cff6bb8f9715dc1'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:34.966470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1236/2515] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:01:34 2022] DELETE /identity/v3/users/f87304ad8e49495a8cff6bb8f9715dc1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:35.236566 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:35.238017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:35.251573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d544b-abe0-45dc-a5ad-5884500c3d0a admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:35.257033 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:35.311073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecc26d-030c-4cf8-bf01-ad009839e1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:35.311900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc26d-030c-4cf8-bf01-ad009839e1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:35.312183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc26d-030c-4cf8-bf01-ad009839e1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:35.313062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecc26d-030c-4cf8-bf01-ad009839e1d8 admin admin] PATH_INFO: `/v3/projects/7a29d6a6864f4c2f965568170a346181` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:35.314248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecc26d-030c-4cf8-bf01-ad009839e1d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a29d6a6864f4c2f965568170a346181)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:35.316274 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecc26d-030c-4cf8-bf01-ad009839e1d8 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:35.360772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-02ecc26d-030c-4cf8-bf01-ad009839e1d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a29d6a6864f4c2f965568170a346181', 'request_id': 'req-02ecc26d-030c-4cf8-bf01-ad009839e1d8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:35.366963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-02ecc26d-030c-4cf8-bf01-ad009839e1d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a29d6a6864f4c2f965568170a346181', 'request_id': 'req-02ecc26d-030c-4cf8-bf01-ad009839e1d8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:35.372404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-02ecc26d-030c-4cf8-bf01-ad009839e1d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a29d6a6864f4c2f965568170a346181', 'request_id': 'req-02ecc26d-030c-4cf8-bf01-ad009839e1d8'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:35.384870 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-02ecc26d-030c-4cf8-bf01-ad009839e1d8 admin admin] The token cache is being invalidate because project 7a29d6a6864f4c2f965568170a346181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:01:35.385158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-02ecc26d-030c-4cf8-bf01-ad009839e1d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:35.385770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-02ecc26d-030c-4cf8-bf01-ad009839e1d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:35.387313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1280/2516] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:01:35 2022] DELETE /identity/v3/projects/7a29d6a6864f4c2f965568170a346181 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:01:35.392899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c234b-2c59-4e09-9566-51f51b6a6da6 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:35.393184 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c234b-2c59-4e09-9566-51f51b6a6da6 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:35.393461 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c234b-2c59-4e09-9566-51f51b6a6da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:35.411886 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-399c234b-2c59-4e09-9566-51f51b6a6da6 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:35.413108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-399c234b-2c59-4e09-9566-51f51b6a6da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:35.473071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1237/2517] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:35.480863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790d5d8-8adf-457e-983a-1e42189c35d3 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:35.481154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790d5d8-8adf-457e-983a-1e42189c35d3 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:35.481429 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790d5d8-8adf-457e-983a-1e42189c35d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:35.500049 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-2790d5d8-8adf-457e-983a-1e42189c35d3 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:35.501258 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-2790d5d8-8adf-457e-983a-1e42189c35d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:35.508981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1281/2518] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:35.513115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c234b-2c59-4e09-9566-51f51b6a6da6 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:35.522974 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c234b-2c59-4e09-9566-51f51b6a6da6 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:35.527574 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4c68e-7cce-498c-bad9-fe1c7a0b8385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:35.528267 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4c68e-7cce-498c-bad9-fe1c7a0b8385 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:35.528551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4c68e-7cce-498c-bad9-fe1c7a0b8385 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:35.528828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4c68e-7cce-498c-bad9-fe1c7a0b8385 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:35.529556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4c68e-7cce-498c-bad9-fe1c7a0b8385 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:01:35.529865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4c68e-7cce-498c-bad9-fe1c7a0b8385 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:35.532280 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4c68e-7cce-498c-bad9-fe1c7a0b8385 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:35.544758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1238/2519] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:01:35.549670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790d5d8-8adf-457e-983a-1e42189c35d3 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:35.558920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790d5d8-8adf-457e-983a-1e42189c35d3 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:35.563756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d284985-cfbd-4309-950e-529ebd5b67e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:35.564371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d284985-cfbd-4309-950e-529ebd5b67e5 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:35.564773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d284985-cfbd-4309-950e-529ebd5b67e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:35.565061 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d284985-cfbd-4309-950e-529ebd5b67e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:35.565763 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d284985-cfbd-4309-950e-529ebd5b67e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:35.567660 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d284985-cfbd-4309-950e-529ebd5b67e5 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:35.592859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1282/2520] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:01:35 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:01:35.597617 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4c68e-7cce-498c-bad9-fe1c7a0b8385 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:35.607554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4c68e-7cce-498c-bad9-fe1c7a0b8385 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:35.663899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6be4d-22d6-4fdc-ae35-466d89441daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:35.664568 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6be4d-22d6-4fdc-ae35-466d89441daf admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:35.664851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6be4d-22d6-4fdc-ae35-466d89441daf admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:35.665127 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6be4d-22d6-4fdc-ae35-466d89441daf admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:35.665824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6be4d-22d6-4fdc-ae35-466d89441daf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:35.667742 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6be4d-22d6-4fdc-ae35-466d89441daf admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:35.686929 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1239/2521] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:01:35 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:01:35.691122 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d284985-cfbd-4309-950e-529ebd5b67e5 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:35.701371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d284985-cfbd-4309-950e-529ebd5b67e5 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:35.706137 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aea4fe-6095-4491-bc9a-2448e6e27faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:35.706791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aea4fe-6095-4491-bc9a-2448e6e27faf admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:35.707326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aea4fe-6095-4491-bc9a-2448e6e27faf admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:35.707326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aea4fe-6095-4491-bc9a-2448e6e27faf admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:35.708077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aea4fe-6095-4491-bc9a-2448e6e27faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:35.709642 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aea4fe-6095-4491-bc9a-2448e6e27faf admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:35.720530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1283/2522] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:35.724426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6be4d-22d6-4fdc-ae35-466d89441daf admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:35.734845 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6be4d-22d6-4fdc-ae35-466d89441daf admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:35.739448 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e0f65-ebe0-4582-ad7e-d7f5805b4b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:35.740397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e0f65-ebe0-4582-ad7e-d7f5805b4b00 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:35.740677 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e0f65-ebe0-4582-ad7e-d7f5805b4b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:35.740948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e0f65-ebe0-4582-ad7e-d7f5805b4b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:35.741663 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e0f65-ebe0-4582-ad7e-d7f5805b4b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:35.743239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e0f65-ebe0-4582-ad7e-d7f5805b4b00 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:35.753016 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1240/2523] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:35.756941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aea4fe-6095-4491-bc9a-2448e6e27faf admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:35.767215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aea4fe-6095-4491-bc9a-2448e6e27faf admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:35.771906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909f3a2-fc48-4ee1-8970-373bb91b2400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:35.772781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909f3a2-fc48-4ee1-8970-373bb91b2400 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:35.773063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909f3a2-fc48-4ee1-8970-373bb91b2400 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:35.773337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909f3a2-fc48-4ee1-8970-373bb91b2400 admin admin] PATH_INFO: `/v3/projects/85697b66d909495ab871786240f7ebde/users/1d67a1ed189b477dbb4e1a2924fb142d/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:35.788385 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909f3a2-fc48-4ee1-8970-373bb91b2400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85697b66d909495ab871786240f7ebde, user_id=1d67a1ed189b477dbb4e1a2924fb142d, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:35.791020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909f3a2-fc48-4ee1-8970-373bb91b2400 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:35.804522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1284/2524] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:01:35 2022] PUT /identity/v3/projects/85697b66d909495ab871786240f7ebde/users/1d67a1ed189b477dbb4e1a2924fb142d/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:35.814683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10013f46-6362-407d-b736-557cb81d589c None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:35.815164 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10013f46-6362-407d-b736-557cb81d589c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:35.815262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10013f46-6362-407d-b736-557cb81d589c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:35.841669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-10013f46-6362-407d-b736-557cb81d589c None None] MFA Rules not processed for user `1d67a1ed189b477dbb4e1a2924fb142d`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:35.842860 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-10013f46-6362-407d-b736-557cb81d589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:35.861324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:35.910816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1241/2525] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:35.915249 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909f3a2-fc48-4ee1-8970-373bb91b2400 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:35.953704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:35.978347 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909f3a2-fc48-4ee1-8970-373bb91b2400 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:35.982993 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c06ed-92da-469d-a953-5e99c235eeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:35.983644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c06ed-92da-469d-a953-5e99c235eeaf admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:35.983957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c06ed-92da-469d-a953-5e99c235eeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:35.984270 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c06ed-92da-469d-a953-5e99c235eeaf admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:35.985290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c06ed-92da-469d-a953-5e99c235eeaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:35.987191 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c06ed-92da-469d-a953-5e99c235eeaf admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:36.012961 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1285/2526] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:01:35 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:01:36.017835 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10013f46-6362-407d-b736-557cb81d589c None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:36.028502 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10013f46-6362-407d-b736-557cb81d589c None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:36.086930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef666a7-eb9c-4e68-9622-e71916c1934d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:36.087610 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef666a7-eb9c-4e68-9622-e71916c1934d admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:36.087932 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef666a7-eb9c-4e68-9622-e71916c1934d admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:36.088214 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef666a7-eb9c-4e68-9622-e71916c1934d admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:36.088906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef666a7-eb9c-4e68-9622-e71916c1934d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:36.090835 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef666a7-eb9c-4e68-9622-e71916c1934d admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:36.111196 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1242/2527] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:01:36 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 21:01:36.115558 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c06ed-92da-469d-a953-5e99c235eeaf admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:36.125945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c06ed-92da-469d-a953-5e99c235eeaf admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:36.130949 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388e662-45e8-4f01-96fc-c227e4a250eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:36.131613 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388e662-45e8-4f01-96fc-c227e4a250eb admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:36.131965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388e662-45e8-4f01-96fc-c227e4a250eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:36.132247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388e662-45e8-4f01-96fc-c227e4a250eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:36.133192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388e662-45e8-4f01-96fc-c227e4a250eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:36.134818 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388e662-45e8-4f01-96fc-c227e4a250eb admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:36.146291 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1286/2528] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:36.150396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef666a7-eb9c-4e68-9622-e71916c1934d admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:36.161195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef666a7-eb9c-4e68-9622-e71916c1934d admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:36.165906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bb7fa-bf2a-4082-929b-2734bdecf505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:36.166785 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bb7fa-bf2a-4082-929b-2734bdecf505 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:36.167068 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bb7fa-bf2a-4082-929b-2734bdecf505 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:36.167348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bb7fa-bf2a-4082-929b-2734bdecf505 admin admin] PATH_INFO: `/v3/projects/315124d32e0d4180bb3c28809f4eb716/users/868c8fa6b86a4712bfde02a0774be7a6/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:36.182285 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bb7fa-bf2a-4082-929b-2734bdecf505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=315124d32e0d4180bb3c28809f4eb716, user_id=868c8fa6b86a4712bfde02a0774be7a6, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:36.184992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bb7fa-bf2a-4082-929b-2734bdecf505 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:36.200542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1243/2529] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:01:36 2022] PUT /identity/v3/projects/315124d32e0d4180bb3c28809f4eb716/users/868c8fa6b86a4712bfde02a0774be7a6/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:36.205377 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388e662-45e8-4f01-96fc-c227e4a250eb admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:36.215590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388e662-45e8-4f01-96fc-c227e4a250eb admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:36.274950 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8f100-b4dd-4e05-84a4-7727b73ae885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:36.275619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8f100-b4dd-4e05-84a4-7727b73ae885 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:36.275932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8f100-b4dd-4e05-84a4-7727b73ae885 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:36.276212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8f100-b4dd-4e05-84a4-7727b73ae885 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:36.276935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8f100-b4dd-4e05-84a4-7727b73ae885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:36.278530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8f100-b4dd-4e05-84a4-7727b73ae885 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:36.288769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1287/2530] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 21:01:36.292895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bb7fa-bf2a-4082-929b-2734bdecf505 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:36.304119 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bb7fa-bf2a-4082-929b-2734bdecf505 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:36.308699 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601de947-28d7-4d2f-a6f4-66571e4422ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:36.309569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601de947-28d7-4d2f-a6f4-66571e4422ca admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:36.309849 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601de947-28d7-4d2f-a6f4-66571e4422ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:36.310124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601de947-28d7-4d2f-a6f4-66571e4422ca admin admin] PATH_INFO: `/v3/projects/315124d32e0d4180bb3c28809f4eb716/users/868c8fa6b86a4712bfde02a0774be7a6/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:36.312123 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601de947-28d7-4d2f-a6f4-66571e4422ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=315124d32e0d4180bb3c28809f4eb716, user_id=868c8fa6b86a4712bfde02a0774be7a6, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:36.315253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601de947-28d7-4d2f-a6f4-66571e4422ca admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:36.325372 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1244/2531] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:01:36 2022] PUT /identity/v3/projects/315124d32e0d4180bb3c28809f4eb716/users/868c8fa6b86a4712bfde02a0774be7a6/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:36.337357 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363c802-ead9-44f4-98ae-437bed6a4f0f None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:36.337589 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363c802-ead9-44f4-98ae-437bed6a4f0f None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:36.337858 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363c802-ead9-44f4-98ae-437bed6a4f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:36.363648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-9363c802-ead9-44f4-98ae-437bed6a4f0f None None] MFA Rules not processed for user `868c8fa6b86a4712bfde02a0774be7a6`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:36.365105 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9363c802-ead9-44f4-98ae-437bed6a4f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:36.434016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1288/2532] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:36.446297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601de947-28d7-4d2f-a6f4-66571e4422ca admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:36.448655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-601de947-28d7-4d2f-a6f4-66571e4422ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:36.508475 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601de947-28d7-4d2f-a6f4-66571e4422ca admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:36.512118 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d45e3-5ec8-46b3-8667-9eb8424e1a96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:36.512752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d45e3-5ec8-46b3-8667-9eb8424e1a96 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:36.513030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d45e3-5ec8-46b3-8667-9eb8424e1a96 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:36.513302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d45e3-5ec8-46b3-8667-9eb8424e1a96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:36.522807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d45e3-5ec8-46b3-8667-9eb8424e1a96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:36.524250 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d45e3-5ec8-46b3-8667-9eb8424e1a96 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:36.533728 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1245/2533] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:36.601794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363c802-ead9-44f4-98ae-437bed6a4f0f None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:36.612487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363c802-ead9-44f4-98ae-437bed6a4f0f None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:36.616452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ab81d-4322-4ba6-9799-92cf46d6da9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:36.617220 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ab81d-4322-4ba6-9799-92cf46d6da9a service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:36.617527 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ab81d-4322-4ba6-9799-92cf46d6da9a service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:36.617774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ab81d-4322-4ba6-9799-92cf46d6da9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:36.649636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:36.684249 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ab81d-4322-4ba6-9799-92cf46d6da9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:36.685714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ab81d-4322-4ba6-9799-92cf46d6da9a service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:36.695900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1289/2534] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:36.952144 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d45e3-5ec8-46b3-8667-9eb8424e1a96 service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:36.969903 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:37.018322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d45e3-5ec8-46b3-8667-9eb8424e1a96 service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:37.022852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9986a0-12d2-45c4-83cc-59e116c83316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:37.023849 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9986a0-12d2-45c4-83cc-59e116c83316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:37.024125 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9986a0-12d2-45c4-83cc-59e116c83316 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:37.024391 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9986a0-12d2-45c4-83cc-59e116c83316 admin admin] PATH_INFO: `/v3/users/1d67a1ed189b477dbb4e1a2924fb142d` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:37.025593 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9986a0-12d2-45c4-83cc-59e116c83316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d67a1ed189b477dbb4e1a2924fb142d)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:37.027522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9986a0-12d2-45c4-83cc-59e116c83316 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:37.074224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-af9986a0-12d2-45c4-83cc-59e116c83316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d67a1ed189b477dbb4e1a2924fb142d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:37.082060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-af9986a0-12d2-45c4-83cc-59e116c83316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d67a1ed189b477dbb4e1a2924fb142d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:37.082755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-af9986a0-12d2-45c4-83cc-59e116c83316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d67a1ed189b477dbb4e1a2924fb142d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:37.089821 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-af9986a0-12d2-45c4-83cc-59e116c83316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d67a1ed189b477dbb4e1a2924fb142d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:37.090494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-af9986a0-12d2-45c4-83cc-59e116c83316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d67a1ed189b477dbb4e1a2924fb142d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:37.107976 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1246/2535] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:01:36 2022] DELETE /identity/v3/users/1d67a1ed189b477dbb4e1a2924fb142d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:01:37.113372 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90918f08-a593-467a-8f5d-f0eea2b7a9d1 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:37.114148 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90918f08-a593-467a-8f5d-f0eea2b7a9d1 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:37.114325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90918f08-a593-467a-8f5d-f0eea2b7a9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:37.131582 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-90918f08-a593-467a-8f5d-f0eea2b7a9d1 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:37.132726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-90918f08-a593-467a-8f5d-f0eea2b7a9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:37.191053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1290/2536] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:37.200814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9986a0-12d2-45c4-83cc-59e116c83316 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:37.202244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-af9986a0-12d2-45c4-83cc-59e116c83316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:37.261336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9986a0-12d2-45c4-83cc-59e116c83316 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:37.265014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecafe33-14ab-4670-a01d-82e21610802e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:37.265752 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecafe33-14ab-4670-a01d-82e21610802e service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:37.266040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecafe33-14ab-4670-a01d-82e21610802e service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:37.266315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecafe33-14ab-4670-a01d-82e21610802e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:37.275398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecafe33-14ab-4670-a01d-82e21610802e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:37.276850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecafe33-14ab-4670-a01d-82e21610802e service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:37.287693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1247/2537] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:37.555794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90918f08-a593-467a-8f5d-f0eea2b7a9d1 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:37.557217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-90918f08-a593-467a-8f5d-f0eea2b7a9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:37.570068 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90918f08-a593-467a-8f5d-f0eea2b7a9d1 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:37.574798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6d24b-e672-44ce-ad41-a95a7e896cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:37.575664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6d24b-e672-44ce-ad41-a95a7e896cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:37.576001 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6d24b-e672-44ce-ad41-a95a7e896cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:37.576280 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6d24b-e672-44ce-ad41-a95a7e896cc1 admin admin] PATH_INFO: `/v3/projects/85697b66d909495ab871786240f7ebde` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:37.577481 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6d24b-e672-44ce-ad41-a95a7e896cc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85697b66d909495ab871786240f7ebde)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:37.580246 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6d24b-e672-44ce-ad41-a95a7e896cc1 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:37.632320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45e6d24b-e672-44ce-ad41-a95a7e896cc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85697b66d909495ab871786240f7ebde', 'request_id': 'req-45e6d24b-e672-44ce-ad41-a95a7e896cc1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:37.638992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45e6d24b-e672-44ce-ad41-a95a7e896cc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85697b66d909495ab871786240f7ebde', 'request_id': 'req-45e6d24b-e672-44ce-ad41-a95a7e896cc1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:37.645326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45e6d24b-e672-44ce-ad41-a95a7e896cc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85697b66d909495ab871786240f7ebde', 'request_id': 'req-45e6d24b-e672-44ce-ad41-a95a7e896cc1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:37.658512 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45e6d24b-e672-44ce-ad41-a95a7e896cc1 admin admin] The token cache is being invalidate because project 85697b66d909495ab871786240f7ebde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:01:37.658807 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45e6d24b-e672-44ce-ad41-a95a7e896cc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:37.659434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45e6d24b-e672-44ce-ad41-a95a7e896cc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:37.661011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1291/2538] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:01:37 2022] DELETE /identity/v3/projects/85697b66d909495ab871786240f7ebde => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:37.665002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecafe33-14ab-4670-a01d-82e21610802e service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:37.674678 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecafe33-14ab-4670-a01d-82e21610802e service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:37.708002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:37.731747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50280d8-9957-4051-9da5-a8ac3ac5e1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:37.732618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50280d8-9957-4051-9da5-a8ac3ac5e1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:37.732898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50280d8-9957-4051-9da5-a8ac3ac5e1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:37.733176 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50280d8-9957-4051-9da5-a8ac3ac5e1f8 admin admin] PATH_INFO: `/v3/users/868c8fa6b86a4712bfde02a0774be7a6` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:37.734426 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50280d8-9957-4051-9da5-a8ac3ac5e1f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=868c8fa6b86a4712bfde02a0774be7a6)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:37.736411 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50280d8-9957-4051-9da5-a8ac3ac5e1f8 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:37.781786 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-f50280d8-9957-4051-9da5-a8ac3ac5e1f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '868c8fa6b86a4712bfde02a0774be7a6'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:37.788840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-f50280d8-9957-4051-9da5-a8ac3ac5e1f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '868c8fa6b86a4712bfde02a0774be7a6'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:37.789452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-f50280d8-9957-4051-9da5-a8ac3ac5e1f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '868c8fa6b86a4712bfde02a0774be7a6'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:37.796191 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-f50280d8-9957-4051-9da5-a8ac3ac5e1f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '868c8fa6b86a4712bfde02a0774be7a6'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:37.797003 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-f50280d8-9957-4051-9da5-a8ac3ac5e1f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '868c8fa6b86a4712bfde02a0774be7a6'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:37.814747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1248/2539] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:01:37 2022] DELETE /identity/v3/users/868c8fa6b86a4712bfde02a0774be7a6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:01:38.091683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6d24b-e672-44ce-ad41-a95a7e896cc1 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:38.093161 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-45e6d24b-e672-44ce-ad41-a95a7e896cc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:38.108221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6d24b-e672-44ce-ad41-a95a7e896cc1 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:38.157213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:38.167649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73120e17-9298-4093-9bcd-22d7344602ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:38.168494 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73120e17-9298-4093-9bcd-22d7344602ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:38.168939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73120e17-9298-4093-9bcd-22d7344602ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:38.169214 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73120e17-9298-4093-9bcd-22d7344602ce admin admin] PATH_INFO: `/v3/projects/315124d32e0d4180bb3c28809f4eb716` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:38.170408 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73120e17-9298-4093-9bcd-22d7344602ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=315124d32e0d4180bb3c28809f4eb716)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:38.172375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73120e17-9298-4093-9bcd-22d7344602ce admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:38.217975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-73120e17-9298-4093-9bcd-22d7344602ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '315124d32e0d4180bb3c28809f4eb716', 'request_id': 'req-73120e17-9298-4093-9bcd-22d7344602ce'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:38.224863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-73120e17-9298-4093-9bcd-22d7344602ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '315124d32e0d4180bb3c28809f4eb716', 'request_id': 'req-73120e17-9298-4093-9bcd-22d7344602ce'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:38.230317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-73120e17-9298-4093-9bcd-22d7344602ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '315124d32e0d4180bb3c28809f4eb716', 'request_id': 'req-73120e17-9298-4093-9bcd-22d7344602ce'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:38.242679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-73120e17-9298-4093-9bcd-22d7344602ce admin admin] The token cache is being invalidate because project 315124d32e0d4180bb3c28809f4eb716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:01:38.242966 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-73120e17-9298-4093-9bcd-22d7344602ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:38.243595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-73120e17-9298-4093-9bcd-22d7344602ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:38.245220 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1292/2540] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:01:38 2022] DELETE /identity/v3/projects/315124d32e0d4180bb3c28809f4eb716 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:38.250255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298aaea9-cae3-47e8-929e-223d761ed030 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:38.250747 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298aaea9-cae3-47e8-929e-223d761ed030 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:38.251112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298aaea9-cae3-47e8-929e-223d761ed030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:38.269547 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-298aaea9-cae3-47e8-929e-223d761ed030 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:38.270755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-298aaea9-cae3-47e8-929e-223d761ed030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:38.330678 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1249/2541] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:38.338383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37a97f-e7bd-431e-8092-56987163bc73 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:38.338673 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37a97f-e7bd-431e-8092-56987163bc73 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:38.338977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37a97f-e7bd-431e-8092-56987163bc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:38.357993 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-de37a97f-e7bd-431e-8092-56987163bc73 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:38.359218 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-de37a97f-e7bd-431e-8092-56987163bc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:38.367344 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1293/2542] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 21:01:38.371475 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298aaea9-cae3-47e8-929e-223d761ed030 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:38.381188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298aaea9-cae3-47e8-929e-223d761ed030 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:38.385772 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22b716-53db-4062-a4f1-22994d36a501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:38.386419 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22b716-53db-4062-a4f1-22994d36a501 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:38.386703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22b716-53db-4062-a4f1-22994d36a501 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:38.386976 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22b716-53db-4062-a4f1-22994d36a501 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:38.387697 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22b716-53db-4062-a4f1-22994d36a501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:01:38.388029 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22b716-53db-4062-a4f1-22994d36a501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:38.389617 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22b716-53db-4062-a4f1-22994d36a501 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:38.401938 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1250/2543] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:01:38.406793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37a97f-e7bd-431e-8092-56987163bc73 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:38.416353 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37a97f-e7bd-431e-8092-56987163bc73 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:38.421226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4e4f3-340f-41cd-be1d-47ad68867d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:38.421805 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4e4f3-340f-41cd-be1d-47ad68867d65 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:38.422091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4e4f3-340f-41cd-be1d-47ad68867d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:38.422364 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4e4f3-340f-41cd-be1d-47ad68867d65 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:38.423058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4e4f3-340f-41cd-be1d-47ad68867d65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:38.425013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4e4f3-340f-41cd-be1d-47ad68867d65 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:38.451415 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1294/2544] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:01:38 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:01:38.456396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22b716-53db-4062-a4f1-22994d36a501 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:38.466329 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22b716-53db-4062-a4f1-22994d36a501 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:38.508977 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:38.525404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4aca7-8e6b-45d3-b5b6-53afcb769d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:38.526070 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4aca7-8e6b-45d3-b5b6-53afcb769d88 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:38.526351 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4aca7-8e6b-45d3-b5b6-53afcb769d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:38.526630 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4aca7-8e6b-45d3-b5b6-53afcb769d88 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:38.527322 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4aca7-8e6b-45d3-b5b6-53afcb769d88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:38.529261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4aca7-8e6b-45d3-b5b6-53afcb769d88 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:38.549415 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1251/2545] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:01:38 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:01:38.553481 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4e4f3-340f-41cd-be1d-47ad68867d65 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:38.564380 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4e4f3-340f-41cd-be1d-47ad68867d65 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:38.569087 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f7ed6-0b26-4f29-be73-afcbd898ed4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:38.569742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f7ed6-0b26-4f29-be73-afcbd898ed4c admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:38.570028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f7ed6-0b26-4f29-be73-afcbd898ed4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:38.570304 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f7ed6-0b26-4f29-be73-afcbd898ed4c admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:38.571029 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f7ed6-0b26-4f29-be73-afcbd898ed4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:38.572650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f7ed6-0b26-4f29-be73-afcbd898ed4c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:38.583884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1295/2546] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:38.587707 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4aca7-8e6b-45d3-b5b6-53afcb769d88 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:38.597652 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4aca7-8e6b-45d3-b5b6-53afcb769d88 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:38.602211 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666600b-1320-4db7-8f12-05ff7d9cfdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:38.602857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666600b-1320-4db7-8f12-05ff7d9cfdcf admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:38.603137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666600b-1320-4db7-8f12-05ff7d9cfdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:38.603409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666600b-1320-4db7-8f12-05ff7d9cfdcf admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:38.604157 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666600b-1320-4db7-8f12-05ff7d9cfdcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:38.605730 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666600b-1320-4db7-8f12-05ff7d9cfdcf admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:38.615810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1252/2547] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:38.619880 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f7ed6-0b26-4f29-be73-afcbd898ed4c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:38.630304 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f7ed6-0b26-4f29-be73-afcbd898ed4c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:38.634980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfaf34-f943-47b0-8f70-d1ceb6eb48b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:38.635933 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfaf34-f943-47b0-8f70-d1ceb6eb48b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:38.636263 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfaf34-f943-47b0-8f70-d1ceb6eb48b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:38.636778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfaf34-f943-47b0-8f70-d1ceb6eb48b5 admin admin] PATH_INFO: `/v3/projects/98091c4f404a420b81a5fc2dac374ad8/users/9cb115c0b89244988c8c191b6b09776f/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:38.652278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfaf34-f943-47b0-8f70-d1ceb6eb48b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98091c4f404a420b81a5fc2dac374ad8, user_id=9cb115c0b89244988c8c191b6b09776f, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:38.655460 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfaf34-f943-47b0-8f70-d1ceb6eb48b5 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:38.668311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1296/2548] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:01:38 2022] PUT /identity/v3/projects/98091c4f404a420b81a5fc2dac374ad8/users/9cb115c0b89244988c8c191b6b09776f/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:38.678995 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92418a98-ca1f-4149-b69c-dc40d68c74cd None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:38.679253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92418a98-ca1f-4149-b69c-dc40d68c74cd None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:38.679539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92418a98-ca1f-4149-b69c-dc40d68c74cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:38.707339 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-92418a98-ca1f-4149-b69c-dc40d68c74cd None None] MFA Rules not processed for user `9cb115c0b89244988c8c191b6b09776f`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:38.708630 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-92418a98-ca1f-4149-b69c-dc40d68c74cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:38.772612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1253/2549] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:38.777154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfaf34-f943-47b0-8f70-d1ceb6eb48b5 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:38.837753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfaf34-f943-47b0-8f70-d1ceb6eb48b5 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:38.842380 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21670c28-7636-4afb-8b0e-fa43ae5b826d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:38.843023 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21670c28-7636-4afb-8b0e-fa43ae5b826d admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:38.843305 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21670c28-7636-4afb-8b0e-fa43ae5b826d admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:38.843608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21670c28-7636-4afb-8b0e-fa43ae5b826d admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:38.844398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21670c28-7636-4afb-8b0e-fa43ae5b826d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:38.846271 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21670c28-7636-4afb-8b0e-fa43ae5b826d admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:38.872038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1297/2550] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:01:38 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:01:38.876760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92418a98-ca1f-4149-b69c-dc40d68c74cd None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:38.887530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92418a98-ca1f-4149-b69c-dc40d68c74cd None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:38.944101 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc066a5-40cb-4e96-9ed8-6618a8f3bca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:38.944709 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc066a5-40cb-4e96-9ed8-6618a8f3bca2 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:38.944999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc066a5-40cb-4e96-9ed8-6618a8f3bca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:38.945279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc066a5-40cb-4e96-9ed8-6618a8f3bca2 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:38.945983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc066a5-40cb-4e96-9ed8-6618a8f3bca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:38.947931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc066a5-40cb-4e96-9ed8-6618a8f3bca2 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:38.970507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1254/2551] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:01:38 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 21:01:38.972379 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21670c28-7636-4afb-8b0e-fa43ae5b826d admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:38.979041 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:38.983074 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21670c28-7636-4afb-8b0e-fa43ae5b826d admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:38.990370 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af511242-a1ca-4731-8f5e-1eb9de369fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:38.991183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af511242-a1ca-4731-8f5e-1eb9de369fea admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:38.991682 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af511242-a1ca-4731-8f5e-1eb9de369fea admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:38.992029 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af511242-a1ca-4731-8f5e-1eb9de369fea admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:38.992826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af511242-a1ca-4731-8f5e-1eb9de369fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:38.994575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af511242-a1ca-4731-8f5e-1eb9de369fea admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:39.006097 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1298/2552] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:39.010371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc066a5-40cb-4e96-9ed8-6618a8f3bca2 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:39.020691 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc066a5-40cb-4e96-9ed8-6618a8f3bca2 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:39.025954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53433c-4ebf-4775-9d4c-7819cca5aa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:39.026829 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53433c-4ebf-4775-9d4c-7819cca5aa97 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:39.027107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53433c-4ebf-4775-9d4c-7819cca5aa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:39.027380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53433c-4ebf-4775-9d4c-7819cca5aa97 admin admin] PATH_INFO: `/v3/projects/5f3f157479354a43a4237937d2bdfb9e/users/6315773163084ea3a76cfbc0ae938380/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:39.042034 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53433c-4ebf-4775-9d4c-7819cca5aa97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3f157479354a43a4237937d2bdfb9e, user_id=6315773163084ea3a76cfbc0ae938380, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:39.044861 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53433c-4ebf-4775-9d4c-7819cca5aa97 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:39.058339 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1255/2553] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:01:39 2022] PUT /identity/v3/projects/5f3f157479354a43a4237937d2bdfb9e/users/6315773163084ea3a76cfbc0ae938380/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:39.063086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af511242-a1ca-4731-8f5e-1eb9de369fea admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:39.073120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af511242-a1ca-4731-8f5e-1eb9de369fea admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:39.130125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128e837-9d76-49ed-bf40-ed45ff8416fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:39.130792 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128e837-9d76-49ed-bf40-ed45ff8416fd admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:39.131081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128e837-9d76-49ed-bf40-ed45ff8416fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:39.131364 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128e837-9d76-49ed-bf40-ed45ff8416fd admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:39.132125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128e837-9d76-49ed-bf40-ed45ff8416fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:39.133732 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128e837-9d76-49ed-bf40-ed45ff8416fd admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:39.143804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1299/2554] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:39.147689 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53433c-4ebf-4775-9d4c-7819cca5aa97 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:39.157954 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53433c-4ebf-4775-9d4c-7819cca5aa97 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:39.162623 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6439535-9e42-4616-84ce-799288f0d634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:39.163503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6439535-9e42-4616-84ce-799288f0d634 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:39.163818 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6439535-9e42-4616-84ce-799288f0d634 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:39.164102 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6439535-9e42-4616-84ce-799288f0d634 admin admin] PATH_INFO: `/v3/projects/5f3f157479354a43a4237937d2bdfb9e/users/6315773163084ea3a76cfbc0ae938380/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:39.166103 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6439535-9e42-4616-84ce-799288f0d634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3f157479354a43a4237937d2bdfb9e, user_id=6315773163084ea3a76cfbc0ae938380, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:39.168771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6439535-9e42-4616-84ce-799288f0d634 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:39.179493 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1256/2555] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:01:39 2022] PUT /identity/v3/projects/5f3f157479354a43a4237937d2bdfb9e/users/6315773163084ea3a76cfbc0ae938380/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:39.191433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdc28e-97cf-4d3a-bdb7-a94b408bb66a None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:39.191950 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdc28e-97cf-4d3a-bdb7-a94b408bb66a None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:39.192045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdc28e-97cf-4d3a-bdb7-a94b408bb66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:39.219670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-04cdc28e-97cf-4d3a-bdb7-a94b408bb66a None None] MFA Rules not processed for user `6315773163084ea3a76cfbc0ae938380`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:39.220935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-04cdc28e-97cf-4d3a-bdb7-a94b408bb66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:39.289612 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1300/2556] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:39.300998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6439535-9e42-4616-84ce-799288f0d634 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:39.302776 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e6439535-9e42-4616-84ce-799288f0d634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:39.309275 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:39.362529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6439535-9e42-4616-84ce-799288f0d634 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:39.366471 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04630825-a068-48f0-8939-b6874592c025 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:39.367122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04630825-a068-48f0-8939-b6874592c025 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:39.367409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04630825-a068-48f0-8939-b6874592c025 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:39.367686 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04630825-a068-48f0-8939-b6874592c025 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:39.427108 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04630825-a068-48f0-8939-b6874592c025 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:39.428560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04630825-a068-48f0-8939-b6874592c025 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:39.437865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1257/2557] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:43.579473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdc28e-97cf-4d3a-bdb7-a94b408bb66a None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:43.580909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-04cdc28e-97cf-4d3a-bdb7-a94b408bb66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:43.592630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdc28e-97cf-4d3a-bdb7-a94b408bb66a None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:43.596195 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6f2aa-6ecd-425f-83b4-39b80a922f36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:43.596819 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6f2aa-6ecd-425f-83b4-39b80a922f36 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:43.597099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6f2aa-6ecd-425f-83b4-39b80a922f36 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:43.597374 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6f2aa-6ecd-425f-83b4-39b80a922f36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:43.606243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6f2aa-6ecd-425f-83b4-39b80a922f36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:43.607642 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6f2aa-6ecd-425f-83b4-39b80a922f36 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:43.617715 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1301/2558] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:48.466124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04630825-a068-48f0-8939-b6874592c025 service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:48.528737 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04630825-a068-48f0-8939-b6874592c025 service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:48.534012 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae91b58-9b2a-4fd3-95cb-b454ebae7dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:48.534994 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae91b58-9b2a-4fd3-95cb-b454ebae7dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:48.535340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae91b58-9b2a-4fd3-95cb-b454ebae7dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:48.535644 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae91b58-9b2a-4fd3-95cb-b454ebae7dc0 admin admin] PATH_INFO: `/v3/users/9cb115c0b89244988c8c191b6b09776f` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:48.537003 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae91b58-9b2a-4fd3-95cb-b454ebae7dc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cb115c0b89244988c8c191b6b09776f)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:48.539188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae91b58-9b2a-4fd3-95cb-b454ebae7dc0 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:48.588281 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9ae91b58-9b2a-4fd3-95cb-b454ebae7dc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cb115c0b89244988c8c191b6b09776f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:48.595670 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9ae91b58-9b2a-4fd3-95cb-b454ebae7dc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cb115c0b89244988c8c191b6b09776f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:48.597014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9ae91b58-9b2a-4fd3-95cb-b454ebae7dc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cb115c0b89244988c8c191b6b09776f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:48.602375 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:48.604468 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9ae91b58-9b2a-4fd3-95cb-b454ebae7dc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cb115c0b89244988c8c191b6b09776f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:48.605440 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9ae91b58-9b2a-4fd3-95cb-b454ebae7dc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cb115c0b89244988c8c191b6b09776f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:48.624087 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1258/2559] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:01:48 2022] DELETE /identity/v3/users/9cb115c0b89244988c8c191b6b09776f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:48.628574 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554a864-1ef0-460e-9f6d-f9816980557c None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:48.629085 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554a864-1ef0-460e-9f6d-f9816980557c None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:48.629201 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554a864-1ef0-460e-9f6d-f9816980557c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:48.647129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-6554a864-1ef0-460e-9f6d-f9816980557c None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:48.648364 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6554a864-1ef0-460e-9f6d-f9816980557c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:48.707306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1302/2560] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 21:01:48.716901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae91b58-9b2a-4fd3-95cb-b454ebae7dc0 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:48.718295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-9ae91b58-9b2a-4fd3-95cb-b454ebae7dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:48.778993 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae91b58-9b2a-4fd3-95cb-b454ebae7dc0 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:48.782693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ac466-5ec5-4293-8814-1ce26e9858c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:48.783516 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ac466-5ec5-4293-8814-1ce26e9858c3 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:48.783978 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ac466-5ec5-4293-8814-1ce26e9858c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:48.784273 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ac466-5ec5-4293-8814-1ce26e9858c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:48.793584 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ac466-5ec5-4293-8814-1ce26e9858c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:48.795148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ac466-5ec5-4293-8814-1ce26e9858c3 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:48.805560 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1259/2561] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:49.071746 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554a864-1ef0-460e-9f6d-f9816980557c None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:49.073262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6554a864-1ef0-460e-9f6d-f9816980557c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:49.080459 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:49.087332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554a864-1ef0-460e-9f6d-f9816980557c None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:49.092279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:49.093166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:49.093449 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:49.093759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b admin admin] PATH_INFO: `/v3/projects/98091c4f404a420b81a5fc2dac374ad8` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:49.095010 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b admin admin] RBAC: Authorizing `identity:delete_project(project_id=98091c4f404a420b81a5fc2dac374ad8)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:49.097820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:49.145207 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98091c4f404a420b81a5fc2dac374ad8', 'request_id': 'req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:49.151299 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98091c4f404a420b81a5fc2dac374ad8', 'request_id': 'req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:49.157011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98091c4f404a420b81a5fc2dac374ad8', 'request_id': 'req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:49.169905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b admin admin] The token cache is being invalidate because project 98091c4f404a420b81a5fc2dac374ad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:01:49.170197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:49.170815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:49.172457 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1303/2562] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:01:49 2022] DELETE /identity/v3/projects/98091c4f404a420b81a5fc2dac374ad8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:49.176983 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ac466-5ec5-4293-8814-1ce26e9858c3 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:49.186625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ac466-5ec5-4293-8814-1ce26e9858c3 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:49.244625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917c7e4-50a7-49fa-b008-81a014028328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:49.245779 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917c7e4-50a7-49fa-b008-81a014028328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:49.246066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917c7e4-50a7-49fa-b008-81a014028328 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:49.246396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917c7e4-50a7-49fa-b008-81a014028328 admin admin] PATH_INFO: `/v3/users/6315773163084ea3a76cfbc0ae938380` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:49.247553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917c7e4-50a7-49fa-b008-81a014028328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6315773163084ea3a76cfbc0ae938380)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:49.249499 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917c7e4-50a7-49fa-b008-81a014028328 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:49.293516 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b917c7e4-50a7-49fa-b008-81a014028328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6315773163084ea3a76cfbc0ae938380'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:49.301102 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b917c7e4-50a7-49fa-b008-81a014028328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6315773163084ea3a76cfbc0ae938380'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:49.301912 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b917c7e4-50a7-49fa-b008-81a014028328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6315773163084ea3a76cfbc0ae938380'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:49.308968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b917c7e4-50a7-49fa-b008-81a014028328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6315773163084ea3a76cfbc0ae938380'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:49.309646 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b917c7e4-50a7-49fa-b008-81a014028328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6315773163084ea3a76cfbc0ae938380'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:49.323699 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:49.328942 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1260/2563] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:01:49 2022] DELETE /identity/v3/users/6315773163084ea3a76cfbc0ae938380 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:49.601058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:49.602522 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:49.619079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3b36e-c0b3-497e-b515-fe6b43a0005b admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:49.676312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66967890-efa2-4ac8-85fa-c560450a770f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:49.677177 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66967890-efa2-4ac8-85fa-c560450a770f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:49.677478 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66967890-efa2-4ac8-85fa-c560450a770f admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:49.677735 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66967890-efa2-4ac8-85fa-c560450a770f admin admin] PATH_INFO: `/v3/projects/5f3f157479354a43a4237937d2bdfb9e` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:49.678920 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66967890-efa2-4ac8-85fa-c560450a770f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f3f157479354a43a4237937d2bdfb9e)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:49.680853 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66967890-efa2-4ac8-85fa-c560450a770f admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:49.726485 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-66967890-efa2-4ac8-85fa-c560450a770f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f3f157479354a43a4237937d2bdfb9e', 'request_id': 'req-66967890-efa2-4ac8-85fa-c560450a770f'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:49.732588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-66967890-efa2-4ac8-85fa-c560450a770f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f3f157479354a43a4237937d2bdfb9e', 'request_id': 'req-66967890-efa2-4ac8-85fa-c560450a770f'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:49.738962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-66967890-efa2-4ac8-85fa-c560450a770f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f3f157479354a43a4237937d2bdfb9e', 'request_id': 'req-66967890-efa2-4ac8-85fa-c560450a770f'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:49.750941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-66967890-efa2-4ac8-85fa-c560450a770f admin admin] The token cache is being invalidate because project 5f3f157479354a43a4237937d2bdfb9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:01:49.751232 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-66967890-efa2-4ac8-85fa-c560450a770f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:49.751902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-66967890-efa2-4ac8-85fa-c560450a770f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:49.753744 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1304/2564] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:01:49 2022] DELETE /identity/v3/projects/5f3f157479354a43a4237937d2bdfb9e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:49.760050 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc12f62-e629-47d5-a512-0fa0b2e60cb4 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:49.760532 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc12f62-e629-47d5-a512-0fa0b2e60cb4 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:49.760647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc12f62-e629-47d5-a512-0fa0b2e60cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:49.779746 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-4fc12f62-e629-47d5-a512-0fa0b2e60cb4 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:49.780993 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-4fc12f62-e629-47d5-a512-0fa0b2e60cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:49.849060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1261/2565] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:49.857419 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64edf2-1308-4ca9-90c7-b4672a66beb9 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:49.857707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64edf2-1308-4ca9-90c7-b4672a66beb9 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:49.857989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64edf2-1308-4ca9-90c7-b4672a66beb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:49.877255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-4d64edf2-1308-4ca9-90c7-b4672a66beb9 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:49.878647 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-4d64edf2-1308-4ca9-90c7-b4672a66beb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:49.886593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1305/2566] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:49.890729 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc12f62-e629-47d5-a512-0fa0b2e60cb4 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:49.900565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc12f62-e629-47d5-a512-0fa0b2e60cb4 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:49.905216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509552df-8063-4f36-8553-31a864554b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:49.905869 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509552df-8063-4f36-8553-31a864554b60 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:49.906154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509552df-8063-4f36-8553-31a864554b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:49.906431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509552df-8063-4f36-8553-31a864554b60 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:49.907155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509552df-8063-4f36-8553-31a864554b60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:01:49.907464 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509552df-8063-4f36-8553-31a864554b60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:49.909588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509552df-8063-4f36-8553-31a864554b60 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:49.921070 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1262/2567] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:01:49.926044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64edf2-1308-4ca9-90c7-b4672a66beb9 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:49.935264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64edf2-1308-4ca9-90c7-b4672a66beb9 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:49.940287 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c045f4a-1fab-4ec4-90e6-667cd14b41ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:49.940938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c045f4a-1fab-4ec4-90e6-667cd14b41ba admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:49.941223 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c045f4a-1fab-4ec4-90e6-667cd14b41ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:49.941502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c045f4a-1fab-4ec4-90e6-667cd14b41ba admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:49.942193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c045f4a-1fab-4ec4-90e6-667cd14b41ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:49.944115 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c045f4a-1fab-4ec4-90e6-667cd14b41ba admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:49.948146 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:49.970412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1306/2568] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:01:49 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:01:49.975042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509552df-8063-4f36-8553-31a864554b60 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:49.984958 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509552df-8063-4f36-8553-31a864554b60 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:50.041101 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266a3f1-6ba9-4273-81ef-a57b35ab6e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:50.041688 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266a3f1-6ba9-4273-81ef-a57b35ab6e37 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:50.041966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266a3f1-6ba9-4273-81ef-a57b35ab6e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:50.042233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266a3f1-6ba9-4273-81ef-a57b35ab6e37 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:50.042915 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266a3f1-6ba9-4273-81ef-a57b35ab6e37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:50.044847 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266a3f1-6ba9-4273-81ef-a57b35ab6e37 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:50.064028 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1263/2569] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:01:49 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:01:50.068408 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c045f4a-1fab-4ec4-90e6-667cd14b41ba admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:50.078658 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c045f4a-1fab-4ec4-90e6-667cd14b41ba admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:50.084367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470383ea-c16b-472b-a3b2-50aa023f3189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:50.085023 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470383ea-c16b-472b-a3b2-50aa023f3189 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:50.085307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470383ea-c16b-472b-a3b2-50aa023f3189 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:50.085586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470383ea-c16b-472b-a3b2-50aa023f3189 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:50.086777 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470383ea-c16b-472b-a3b2-50aa023f3189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:50.089123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470383ea-c16b-472b-a3b2-50aa023f3189 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:50.099866 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1307/2570] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:50.103789 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266a3f1-6ba9-4273-81ef-a57b35ab6e37 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:50.115987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266a3f1-6ba9-4273-81ef-a57b35ab6e37 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:50.120717 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa3f65-e355-4b69-af01-52438c124ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:50.121377 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa3f65-e355-4b69-af01-52438c124ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:50.121662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa3f65-e355-4b69-af01-52438c124ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:50.121940 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa3f65-e355-4b69-af01-52438c124ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:50.122663 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa3f65-e355-4b69-af01-52438c124ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:50.124293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa3f65-e355-4b69-af01-52438c124ea3 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:50.134955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1264/2571] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:50.139231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470383ea-c16b-472b-a3b2-50aa023f3189 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:50.149768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470383ea-c16b-472b-a3b2-50aa023f3189 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:50.154426 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c949657-6e62-4fe1-90d4-a1d0c23a7b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:50.155300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c949657-6e62-4fe1-90d4-a1d0c23a7b4e admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:50.155587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c949657-6e62-4fe1-90d4-a1d0c23a7b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:50.155893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c949657-6e62-4fe1-90d4-a1d0c23a7b4e admin admin] PATH_INFO: `/v3/projects/ab9a88f75f5047c48431cd8eda7450cc/users/e4bae420b2f64d4ba5be71f022cc5960/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:50.171302 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c949657-6e62-4fe1-90d4-a1d0c23a7b4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab9a88f75f5047c48431cd8eda7450cc, user_id=e4bae420b2f64d4ba5be71f022cc5960, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:50.174047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c949657-6e62-4fe1-90d4-a1d0c23a7b4e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:50.186921 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1308/2572] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:01:50 2022] PUT /identity/v3/projects/ab9a88f75f5047c48431cd8eda7450cc/users/e4bae420b2f64d4ba5be71f022cc5960/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:50.201103 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7152832-3f8a-4542-b39d-79daca6b85d1 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:50.201423 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7152832-3f8a-4542-b39d-79daca6b85d1 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:50.201739 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7152832-3f8a-4542-b39d-79daca6b85d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:50.229128 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-b7152832-3f8a-4542-b39d-79daca6b85d1 None None] MFA Rules not processed for user `e4bae420b2f64d4ba5be71f022cc5960`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:50.230344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b7152832-3f8a-4542-b39d-79daca6b85d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:50.287226 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:50.297078 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1265/2573] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 21:01:50.308474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c949657-6e62-4fe1-90d4-a1d0c23a7b4e admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:50.310162 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-3c949657-6e62-4fe1-90d4-a1d0c23a7b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:50.370010 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c949657-6e62-4fe1-90d4-a1d0c23a7b4e admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:50.373702 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac72479-57af-4c25-9763-4c85ed401b3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:50.374342 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac72479-57af-4c25-9763-4c85ed401b3d service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:50.374626 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac72479-57af-4c25-9763-4c85ed401b3d service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:50.374906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac72479-57af-4c25-9763-4c85ed401b3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:50.384088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac72479-57af-4c25-9763-4c85ed401b3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:50.385515 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac72479-57af-4c25-9763-4c85ed401b3d service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:50.395021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1309/2574] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:50.413635 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7152832-3f8a-4542-b39d-79daca6b85d1 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:50.474260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7152832-3f8a-4542-b39d-79daca6b85d1 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:50.478854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4031101-e8e3-4c9f-8815-2ab89acee50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:50.479790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4031101-e8e3-4c9f-8815-2ab89acee50f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:50.480011 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4031101-e8e3-4c9f-8815-2ab89acee50f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:50.480289 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4031101-e8e3-4c9f-8815-2ab89acee50f admin admin] PATH_INFO: `/v3/users/e4bae420b2f64d4ba5be71f022cc5960` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:50.481478 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4031101-e8e3-4c9f-8815-2ab89acee50f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4bae420b2f64d4ba5be71f022cc5960)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:50.483391 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4031101-e8e3-4c9f-8815-2ab89acee50f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:50.530488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-e4031101-e8e3-4c9f-8815-2ab89acee50f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4bae420b2f64d4ba5be71f022cc5960'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:50.538263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-e4031101-e8e3-4c9f-8815-2ab89acee50f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4bae420b2f64d4ba5be71f022cc5960'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:50.538931 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-e4031101-e8e3-4c9f-8815-2ab89acee50f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4bae420b2f64d4ba5be71f022cc5960'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:50.546019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-e4031101-e8e3-4c9f-8815-2ab89acee50f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4bae420b2f64d4ba5be71f022cc5960'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:50.546690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-e4031101-e8e3-4c9f-8815-2ab89acee50f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4bae420b2f64d4ba5be71f022cc5960'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:50.564338 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1266/2575] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:01:50 2022] DELETE /identity/v3/users/e4bae420b2f64d4ba5be71f022cc5960 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:50.570391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b5d75-fde0-41a3-accf-bf97ef593250 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:50.571528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b5d75-fde0-41a3-accf-bf97ef593250 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:50.571528 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b5d75-fde0-41a3-accf-bf97ef593250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:50.588474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-fe4b5d75-fde0-41a3-accf-bf97ef593250 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:50.589689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fe4b5d75-fde0-41a3-accf-bf97ef593250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:50.649981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1310/2576] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:50.659612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4031101-e8e3-4c9f-8815-2ab89acee50f admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:50.661058 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e4031101-e8e3-4c9f-8815-2ab89acee50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:50.717533 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4031101-e8e3-4c9f-8815-2ab89acee50f admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:50.721368 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b91b5c-c6c7-4379-b113-71afc16e5f9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:50.722010 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b91b5c-c6c7-4379-b113-71afc16e5f9b service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:50.722305 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b91b5c-c6c7-4379-b113-71afc16e5f9b service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:50.722575 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b91b5c-c6c7-4379-b113-71afc16e5f9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:50.731526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b91b5c-c6c7-4379-b113-71afc16e5f9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:50.732988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b91b5c-c6c7-4379-b113-71afc16e5f9b service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:50.743206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1267/2577] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:51.011083 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b5d75-fde0-41a3-accf-bf97ef593250 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:51.012609 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fe4b5d75-fde0-41a3-accf-bf97ef593250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:51.025322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b5d75-fde0-41a3-accf-bf97ef593250 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:51.030135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0849d-a903-420c-a402-59b91d933b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:51.031006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0849d-a903-420c-a402-59b91d933b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:51.031291 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0849d-a903-420c-a402-59b91d933b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:51.031568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0849d-a903-420c-a402-59b91d933b78 admin admin] PATH_INFO: `/v3/projects/ab9a88f75f5047c48431cd8eda7450cc` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:51.032796 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0849d-a903-420c-a402-59b91d933b78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab9a88f75f5047c48431cd8eda7450cc)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:51.035390 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0849d-a903-420c-a402-59b91d933b78 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:51.082730 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8de0849d-a903-420c-a402-59b91d933b78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab9a88f75f5047c48431cd8eda7450cc', 'request_id': 'req-8de0849d-a903-420c-a402-59b91d933b78'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:51.083842 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:51.090320 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8de0849d-a903-420c-a402-59b91d933b78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab9a88f75f5047c48431cd8eda7450cc', 'request_id': 'req-8de0849d-a903-420c-a402-59b91d933b78'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:51.097152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8de0849d-a903-420c-a402-59b91d933b78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab9a88f75f5047c48431cd8eda7450cc', 'request_id': 'req-8de0849d-a903-420c-a402-59b91d933b78'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:51.111037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8de0849d-a903-420c-a402-59b91d933b78 admin admin] The token cache is being invalidate because project ab9a88f75f5047c48431cd8eda7450cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:01:51.111599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8de0849d-a903-420c-a402-59b91d933b78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:51.112406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-8de0849d-a903-420c-a402-59b91d933b78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:51.113985 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1311/2578] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:01:51 2022] DELETE /identity/v3/projects/ab9a88f75f5047c48431cd8eda7450cc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:51.119521 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dac168-c3dd-4412-b57e-23f1d1258ae5 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:51.120068 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dac168-c3dd-4412-b57e-23f1d1258ae5 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:51.120394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dac168-c3dd-4412-b57e-23f1d1258ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:51.138827 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-a2dac168-c3dd-4412-b57e-23f1d1258ae5 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:51.140090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-a2dac168-c3dd-4412-b57e-23f1d1258ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:51.153024 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:51.202564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1268/2579] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:51.210719 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc4f07-3e2c-4e4f-8fab-91f4e99f937e None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:51.211013 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc4f07-3e2c-4e4f-8fab-91f4e99f937e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:51.211293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc4f07-3e2c-4e4f-8fab-91f4e99f937e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:51.229376 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d4bc4f07-3e2c-4e4f-8fab-91f4e99f937e None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:51.230604 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d4bc4f07-3e2c-4e4f-8fab-91f4e99f937e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:51.238765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1312/2580] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:51.243304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dac168-c3dd-4412-b57e-23f1d1258ae5 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:51.253571 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dac168-c3dd-4412-b57e-23f1d1258ae5 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:51.258165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e4f87-4d88-4bc7-b563-b093f33db66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:51.258810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e4f87-4d88-4bc7-b563-b093f33db66e admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:51.259089 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e4f87-4d88-4bc7-b563-b093f33db66e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:51.259359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e4f87-4d88-4bc7-b563-b093f33db66e admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:51.260103 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e4f87-4d88-4bc7-b563-b093f33db66e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:01:51.260403 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e4f87-4d88-4bc7-b563-b093f33db66e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:51.261996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e4f87-4d88-4bc7-b563-b093f33db66e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:51.274290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1269/2581] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:01:51.278552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc4f07-3e2c-4e4f-8fab-91f4e99f937e None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:51.288212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc4f07-3e2c-4e4f-8fab-91f4e99f937e None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:51.292918 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f625f-654c-40b8-91e5-a668e931bf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:51.293564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f625f-654c-40b8-91e5-a668e931bf5c admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:51.293850 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f625f-654c-40b8-91e5-a668e931bf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:51.294128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f625f-654c-40b8-91e5-a668e931bf5c admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:51.294816 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f625f-654c-40b8-91e5-a668e931bf5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:51.296769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f625f-654c-40b8-91e5-a668e931bf5c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:51.322845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1313/2582] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:01:51 2022] POST /identity/v3/projects => generated 361 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:01:51.326745 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e4f87-4d88-4bc7-b563-b093f33db66e admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:51.338090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e4f87-4d88-4bc7-b563-b093f33db66e admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:51.395523 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be0a1a-c006-4bcb-abef-f6319b543eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:51.396571 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be0a1a-c006-4bcb-abef-f6319b543eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:51.396857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be0a1a-c006-4bcb-abef-f6319b543eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:51.397132 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be0a1a-c006-4bcb-abef-f6319b543eb8 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:51.397828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be0a1a-c006-4bcb-abef-f6319b543eb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:51.399759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be0a1a-c006-4bcb-abef-f6319b543eb8 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:51.419959 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1270/2583] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:01:51 2022] POST /identity/v3/users => generated 281 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:01:51.424063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f625f-654c-40b8-91e5-a668e931bf5c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:51.434726 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f625f-654c-40b8-91e5-a668e931bf5c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:51.440520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20dca8-96b1-4677-8cd9-f252021dc15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:51.441231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20dca8-96b1-4677-8cd9-f252021dc15c admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:51.441518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20dca8-96b1-4677-8cd9-f252021dc15c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:51.441799 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20dca8-96b1-4677-8cd9-f252021dc15c admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:51.442553 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20dca8-96b1-4677-8cd9-f252021dc15c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:51.444173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20dca8-96b1-4677-8cd9-f252021dc15c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:51.454666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1314/2584] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:51.459458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be0a1a-c006-4bcb-abef-f6319b543eb8 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:51.469405 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be0a1a-c006-4bcb-abef-f6319b543eb8 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:51.474075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059436b-1567-405b-8893-1f4ebd008b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:51.474731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059436b-1567-405b-8893-1f4ebd008b96 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:51.475014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059436b-1567-405b-8893-1f4ebd008b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:51.475290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059436b-1567-405b-8893-1f4ebd008b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:51.476058 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059436b-1567-405b-8893-1f4ebd008b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:51.477648 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059436b-1567-405b-8893-1f4ebd008b96 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:51.488046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1271/2585] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:51.492501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20dca8-96b1-4677-8cd9-f252021dc15c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:51.503045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20dca8-96b1-4677-8cd9-f252021dc15c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:51.507871 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401814ad-83c6-42aa-b610-bafe9ecfdecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:51.508756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401814ad-83c6-42aa-b610-bafe9ecfdecd admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:51.509064 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401814ad-83c6-42aa-b610-bafe9ecfdecd admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:51.509371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401814ad-83c6-42aa-b610-bafe9ecfdecd admin admin] PATH_INFO: `/v3/projects/a5d8b42d5f9f442196a57daf127a4230/users/f186a3550a814ca4b7715d4cd581e4b7/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:51.523873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401814ad-83c6-42aa-b610-bafe9ecfdecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5d8b42d5f9f442196a57daf127a4230, user_id=f186a3550a814ca4b7715d4cd581e4b7, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:51.526499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401814ad-83c6-42aa-b610-bafe9ecfdecd admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:51.539587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1315/2586] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:01:51 2022] PUT /identity/v3/projects/a5d8b42d5f9f442196a57daf127a4230/users/f186a3550a814ca4b7715d4cd581e4b7/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:01:51.551116 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee3639-f306-4398-85f2-b97c695d713e None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:51.551409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee3639-f306-4398-85f2-b97c695d713e None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:51.551693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee3639-f306-4398-85f2-b97c695d713e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:51.578567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d4ee3639-f306-4398-85f2-b97c695d713e None None] MFA Rules not processed for user `f186a3550a814ca4b7715d4cd581e4b7`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:51.580100 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d4ee3639-f306-4398-85f2-b97c695d713e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:51.643858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1272/2587] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:51.675202 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401814ad-83c6-42aa-b610-bafe9ecfdecd admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:51.676967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-401814ad-83c6-42aa-b610-bafe9ecfdecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:51.733202 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401814ad-83c6-42aa-b610-bafe9ecfdecd admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:51.737756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bce0ea-e74e-44d2-926f-e1c10ebd3da2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:51.738396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bce0ea-e74e-44d2-926f-e1c10ebd3da2 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:51.738681 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bce0ea-e74e-44d2-926f-e1c10ebd3da2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:51.738959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bce0ea-e74e-44d2-926f-e1c10ebd3da2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:51.747461 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bce0ea-e74e-44d2-926f-e1c10ebd3da2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:51.748927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bce0ea-e74e-44d2-926f-e1c10ebd3da2 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:51.758452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1316/2588] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 21:01:51.772388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee3639-f306-4398-85f2-b97c695d713e None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:51.785631 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:51.836526 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee3639-f306-4398-85f2-b97c695d713e None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:51.842093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fb5b6-b383-45e6-af1f-e197a6ff7ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:51.843043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fb5b6-b383-45e6-af1f-e197a6ff7ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:51.843326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fb5b6-b383-45e6-af1f-e197a6ff7ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:51.843601 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fb5b6-b383-45e6-af1f-e197a6ff7ceb admin admin] PATH_INFO: `/v3/users/f186a3550a814ca4b7715d4cd581e4b7` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:51.844856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fb5b6-b383-45e6-af1f-e197a6ff7ceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f186a3550a814ca4b7715d4cd581e4b7)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:51.847148 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fb5b6-b383-45e6-af1f-e197a6ff7ceb admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:51.895550 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-de5fb5b6-b383-45e6-af1f-e197a6ff7ceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f186a3550a814ca4b7715d4cd581e4b7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:51.902645 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-de5fb5b6-b383-45e6-af1f-e197a6ff7ceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f186a3550a814ca4b7715d4cd581e4b7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:51.903192 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-de5fb5b6-b383-45e6-af1f-e197a6ff7ceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f186a3550a814ca4b7715d4cd581e4b7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:51.910570 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-de5fb5b6-b383-45e6-af1f-e197a6ff7ceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f186a3550a814ca4b7715d4cd581e4b7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:51.911235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-de5fb5b6-b383-45e6-af1f-e197a6ff7ceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f186a3550a814ca4b7715d4cd581e4b7'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:51.928503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1273/2589] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:01:51 2022] DELETE /identity/v3/users/f186a3550a814ca4b7715d4cd581e4b7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:51.933005 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7851b0c-a5a8-4136-b61b-7575e7e9bf7d None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:51.933301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7851b0c-a5a8-4136-b61b-7575e7e9bf7d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:51.933587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7851b0c-a5a8-4136-b61b-7575e7e9bf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:51.950838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-c7851b0c-a5a8-4136-b61b-7575e7e9bf7d None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:51.952093 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c7851b0c-a5a8-4136-b61b-7575e7e9bf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:52.011697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1317/2590] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:52.021675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fb5b6-b383-45e6-af1f-e197a6ff7ceb admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:52.023073 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-de5fb5b6-b383-45e6-af1f-e197a6ff7ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:52.082378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fb5b6-b383-45e6-af1f-e197a6ff7ceb admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:52.086101 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d861f3-36d2-49b7-a177-6436afcc6674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:52.086748 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d861f3-36d2-49b7-a177-6436afcc6674 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:52.087037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d861f3-36d2-49b7-a177-6436afcc6674 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:52.087320 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d861f3-36d2-49b7-a177-6436afcc6674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:52.097483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d861f3-36d2-49b7-a177-6436afcc6674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:52.098914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d861f3-36d2-49b7-a177-6436afcc6674 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:52.108607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1274/2591] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:52.380927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7851b0c-a5a8-4136-b61b-7575e7e9bf7d None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:52.382439 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c7851b0c-a5a8-4136-b61b-7575e7e9bf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:52.404665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7851b0c-a5a8-4136-b61b-7575e7e9bf7d None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:52.410931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67615c-8a0e-4e98-9186-8087a6986002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:52.411981 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67615c-8a0e-4e98-9186-8087a6986002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:52.412332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67615c-8a0e-4e98-9186-8087a6986002 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:52.412674 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67615c-8a0e-4e98-9186-8087a6986002 admin admin] PATH_INFO: `/v3/projects/a5d8b42d5f9f442196a57daf127a4230` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:52.413950 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67615c-8a0e-4e98-9186-8087a6986002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5d8b42d5f9f442196a57daf127a4230)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:52.415849 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67615c-8a0e-4e98-9186-8087a6986002 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:52.428376 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:52.464939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1a67615c-8a0e-4e98-9186-8087a6986002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5d8b42d5f9f442196a57daf127a4230', 'request_id': 'req-1a67615c-8a0e-4e98-9186-8087a6986002'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:52.471743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1a67615c-8a0e-4e98-9186-8087a6986002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5d8b42d5f9f442196a57daf127a4230', 'request_id': 'req-1a67615c-8a0e-4e98-9186-8087a6986002'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:52.477361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1a67615c-8a0e-4e98-9186-8087a6986002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5d8b42d5f9f442196a57daf127a4230', 'request_id': 'req-1a67615c-8a0e-4e98-9186-8087a6986002'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:52.489721 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1a67615c-8a0e-4e98-9186-8087a6986002 admin admin] The token cache is being invalidate because project a5d8b42d5f9f442196a57daf127a4230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:01:52.490384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1a67615c-8a0e-4e98-9186-8087a6986002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:52.491111 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-1a67615c-8a0e-4e98-9186-8087a6986002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:01:52.492609 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1318/2592] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:01:52 2022] DELETE /identity/v3/projects/a5d8b42d5f9f442196a57daf127a4230 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:52.497466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d4835-b2c5-4b39-8dbd-88e9bea83d92 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:52.497922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d4835-b2c5-4b39-8dbd-88e9bea83d92 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:52.498015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d4835-b2c5-4b39-8dbd-88e9bea83d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:52.515200 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-fd8d4835-b2c5-4b39-8dbd-88e9bea83d92 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:52.516414 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-fd8d4835-b2c5-4b39-8dbd-88e9bea83d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:52.547048 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:52.577177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1275/2593] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:52.587940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cc97d-65cb-4008-8524-a93c88b376a2 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:52.588230 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cc97d-65cb-4008-8524-a93c88b376a2 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:52.588541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cc97d-65cb-4008-8524-a93c88b376a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:52.607445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e66cc97d-65cb-4008-8524-a93c88b376a2 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:52.608679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e66cc97d-65cb-4008-8524-a93c88b376a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:52.616765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1319/2594] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 21:01:52.620943 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d4835-b2c5-4b39-8dbd-88e9bea83d92 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:52.631286 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d4835-b2c5-4b39-8dbd-88e9bea83d92 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:52.635949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68cd39-5246-47eb-81ca-0e587009601c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:52.636604 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68cd39-5246-47eb-81ca-0e587009601c admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:52.636894 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68cd39-5246-47eb-81ca-0e587009601c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:52.637174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68cd39-5246-47eb-81ca-0e587009601c admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:52.637902 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68cd39-5246-47eb-81ca-0e587009601c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:01:52.638211 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68cd39-5246-47eb-81ca-0e587009601c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:52.639854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68cd39-5246-47eb-81ca-0e587009601c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:52.651997 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1276/2595] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:01:52.709498 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cc97d-65cb-4008-8524-a93c88b376a2 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:52.718828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cc97d-65cb-4008-8524-a93c88b376a2 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:52.724576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a938490-7f49-4beb-b477-f4ce740a347f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:52.725226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a938490-7f49-4beb-b477-f4ce740a347f admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:52.725512 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a938490-7f49-4beb-b477-f4ce740a347f admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:52.725791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a938490-7f49-4beb-b477-f4ce740a347f admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:52.726483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a938490-7f49-4beb-b477-f4ce740a347f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:52.728416 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a938490-7f49-4beb-b477-f4ce740a347f admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:52.754404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1320/2596] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:01:52 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:01:52.758537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68cd39-5246-47eb-81ca-0e587009601c admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:52.768439 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68cd39-5246-47eb-81ca-0e587009601c admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:52.823529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27917e2-1d49-41d5-bf89-29069bc0b326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:52.824230 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27917e2-1d49-41d5-bf89-29069bc0b326 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:52.824511 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27917e2-1d49-41d5-bf89-29069bc0b326 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:52.824783 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27917e2-1d49-41d5-bf89-29069bc0b326 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:52.825470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27917e2-1d49-41d5-bf89-29069bc0b326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:52.827472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27917e2-1d49-41d5-bf89-29069bc0b326 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:52.847520 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1277/2597] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:01:52 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 21:01:52.851766 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a938490-7f49-4beb-b477-f4ce740a347f admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:52.862637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a938490-7f49-4beb-b477-f4ce740a347f admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:52.867461 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904a1e4-8316-4d84-9c78-b4cf8844e9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:52.868137 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904a1e4-8316-4d84-9c78-b4cf8844e9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:52.868441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904a1e4-8316-4d84-9c78-b4cf8844e9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:52.868715 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904a1e4-8316-4d84-9c78-b4cf8844e9d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:52.869437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4904a1e4-8316-4d84-9c78-b4cf8844e9d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:52.871034 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4904a1e4-8316-4d84-9c78-b4cf8844e9d1 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:52.881502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1321/2598] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:52.885417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27917e2-1d49-41d5-bf89-29069bc0b326 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:52.895897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27917e2-1d49-41d5-bf89-29069bc0b326 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:52.900863 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b507530-3fd7-4487-969e-f942a26fd788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:52.901524 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b507530-3fd7-4487-969e-f942a26fd788 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:52.901810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b507530-3fd7-4487-969e-f942a26fd788 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:52.902086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b507530-3fd7-4487-969e-f942a26fd788 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:52.902815 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b507530-3fd7-4487-969e-f942a26fd788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:52.904444 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b507530-3fd7-4487-969e-f942a26fd788 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:52.915146 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1278/2599] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:01:52.919325 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904a1e4-8316-4d84-9c78-b4cf8844e9d1 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:52.929779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904a1e4-8316-4d84-9c78-b4cf8844e9d1 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:52.934351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5aab84-bc6c-4135-bddc-ecbf1617abe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:52.935208 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5aab84-bc6c-4135-bddc-ecbf1617abe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:52.935487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5aab84-bc6c-4135-bddc-ecbf1617abe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:52.935787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5aab84-bc6c-4135-bddc-ecbf1617abe1 admin admin] PATH_INFO: `/v3/projects/99bf599415154f1eb8ebdf982099e635/users/356d7835faeb4ea09cf7580f5f467473/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:52.949909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5aab84-bc6c-4135-bddc-ecbf1617abe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99bf599415154f1eb8ebdf982099e635, user_id=356d7835faeb4ea09cf7580f5f467473, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:52.952556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5aab84-bc6c-4135-bddc-ecbf1617abe1 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:52.965129 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1322/2600] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:01:52 2022] PUT /identity/v3/projects/99bf599415154f1eb8ebdf982099e635/users/356d7835faeb4ea09cf7580f5f467473/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:01:52.970170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4597ea3-ae7a-419c-8d5b-e36113dff44d None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:52.970466 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4597ea3-ae7a-419c-8d5b-e36113dff44d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:52.971309 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4597ea3-ae7a-419c-8d5b-e36113dff44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:52.989744 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-f4597ea3-ae7a-419c-8d5b-e36113dff44d None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:52.991227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f4597ea3-ae7a-419c-8d5b-e36113dff44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:53.051573 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1279/2601] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 21:01:53.061197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5aab84-bc6c-4135-bddc-ecbf1617abe1 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:53.062899 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ca5aab84-bc6c-4135-bddc-ecbf1617abe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:53.120510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5aab84-bc6c-4135-bddc-ecbf1617abe1 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:53.124240 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb578e8-a098-4dd0-904b-9ec595bae3d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:53.124883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb578e8-a098-4dd0-904b-9ec595bae3d5 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:53.125169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb578e8-a098-4dd0-904b-9ec595bae3d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:53.125453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb578e8-a098-4dd0-904b-9ec595bae3d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:53.134669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb578e8-a098-4dd0-904b-9ec595bae3d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:53.136128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb578e8-a098-4dd0-904b-9ec595bae3d5 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:53.147288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1323/2602] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:53.849063 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2cd83-567d-418c-ad9c-45d96aaad0a0 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:53.849644 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2cd83-567d-418c-ad9c-45d96aaad0a0 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:53.849644 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2cd83-567d-418c-ad9c-45d96aaad0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:53.883577 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-00f2cd83-567d-418c-ad9c-45d96aaad0a0 None None] MFA Rules not processed for user `356d7835faeb4ea09cf7580f5f467473`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:53.884895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-00f2cd83-567d-418c-ad9c-45d96aaad0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:53.916928 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:53.964767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1280/2603] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:01:53.978103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb578e8-a098-4dd0-904b-9ec595bae3d5 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:01:53.979536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8cb578e8-a098-4dd0-904b-9ec595bae3d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:54.033140 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:01:54.051047 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb578e8-a098-4dd0-904b-9ec595bae3d5 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:01:54.055306 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab053ddc-b5e2-46e2-8c02-742bcbf37051 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:01:54.056002 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab053ddc-b5e2-46e2-8c02-742bcbf37051 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:54.056284 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab053ddc-b5e2-46e2-8c02-742bcbf37051 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:54.056562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab053ddc-b5e2-46e2-8c02-742bcbf37051 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:54.067294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab053ddc-b5e2-46e2-8c02-742bcbf37051 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:01:54.069803 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab053ddc-b5e2-46e2-8c02-742bcbf37051 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:01:54.084011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1324/2604] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:01:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:01:55.434508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad98d38-4fa8-4638-9fa5-0b2218a4de44 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:01:55.435032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad98d38-4fa8-4638-9fa5-0b2218a4de44 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:01:55.435140 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad98d38-4fa8-4638-9fa5-0b2218a4de44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:01:55.452561 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-4ad98d38-4fa8-4638-9fa5-0b2218a4de44 None None] MFA Rules not processed for user `356d7835faeb4ea09cf7580f5f467473`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:01:55.453782 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-4ad98d38-4fa8-4638-9fa5-0b2218a4de44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:01:55.461371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1281/2605] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:01:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:02:04.078369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=8 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:02:05.447367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=16 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:02:13.503698 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c33b7b-05fa-4174-b5a3-d1ea8600b4ed None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:02:13.504255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c33b7b-05fa-4174-b5a3-d1ea8600b4ed None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:02:13.504552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c33b7b-05fa-4174-b5a3-d1ea8600b4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:02:13.525527 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-84c33b7b-05fa-4174-b5a3-d1ea8600b4ed None None] MFA Rules not processed for user `356d7835faeb4ea09cf7580f5f467473`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:02:13.526714 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-84c33b7b-05fa-4174-b5a3-d1ea8600b4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:02:13.534331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1325/2606] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:02:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:02:15.564128 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a89483-232d-433a-820a-dc4c1e663aea None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:02:15.564647 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a89483-232d-433a-820a-dc4c1e663aea None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:02:15.564930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a89483-232d-433a-820a-dc4c1e663aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:02:15.587104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-22a89483-232d-433a-820a-dc4c1e663aea None None] MFA Rules not processed for user `ab2bc1eb1eb845ec82b79adee04a51e7`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:02:15.588350 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-22a89483-232d-433a-820a-dc4c1e663aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:02:15.654344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1282/2607] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:02:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:02:23.519154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:02:25.646446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=25 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:02:28.248762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c33b7b-05fa-4174-b5a3-d1ea8600b4ed None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:02:28.250495 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-84c33b7b-05fa-4174-b5a3-d1ea8600b4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:02:28.263524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c33b7b-05fa-4174-b5a3-d1ea8600b4ed None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:02:28.267428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7b173-2c3a-4613-b9b4-097860ac73fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:02:28.268124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7b173-2c3a-4613-b9b4-097860ac73fb service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:02:28.268415 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7b173-2c3a-4613-b9b4-097860ac73fb service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:02:28.268694 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7b173-2c3a-4613-b9b4-097860ac73fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:02:28.270636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-bfe7b173-2c3a-4613-b9b4-097860ac73fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:02:28.348963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7b173-2c3a-4613-b9b4-097860ac73fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:02:28.350588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7b173-2c3a-4613-b9b4-097860ac73fb service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:02:28.361757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1326/2608] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:02:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:02:28.669215 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a89483-232d-433a-820a-dc4c1e663aea None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:02:28.679683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a89483-232d-433a-820a-dc4c1e663aea None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:02:28.683297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348a388-e65b-4a1e-a11e-72aea20ad189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:02:28.683977 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348a388-e65b-4a1e-a11e-72aea20ad189 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:02:28.684265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348a388-e65b-4a1e-a11e-72aea20ad189 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:02:28.684541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348a388-e65b-4a1e-a11e-72aea20ad189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:02:28.686345 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c348a388-e65b-4a1e-a11e-72aea20ad189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:02:28.700187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348a388-e65b-4a1e-a11e-72aea20ad189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:02:28.701615 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348a388-e65b-4a1e-a11e-72aea20ad189 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:02:28.714567 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1283/2609] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:02:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:02:30.289258 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700bb97-d1f0-409f-9581-421150dcce8c None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:02:30.289819 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700bb97-d1f0-409f-9581-421150dcce8c None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:02:30.289819 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700bb97-d1f0-409f-9581-421150dcce8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:02:30.318995 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-c700bb97-d1f0-409f-9581-421150dcce8c None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:02:30.320331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c700bb97-d1f0-409f-9581-421150dcce8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:02:30.381136 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1327/2610] 10.222.0.60 () {58 vars in 1009 bytes} [Thu Jul 21 21:02:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:02:30.430354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348a388-e65b-4a1e-a11e-72aea20ad189 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:02:30.431766 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c348a388-e65b-4a1e-a11e-72aea20ad189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:02:30.505666 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348a388-e65b-4a1e-a11e-72aea20ad189 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:02:30.510761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6dd02-7193-4862-8210-99c52ddd0f6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:02:30.511406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6dd02-7193-4862-8210-99c52ddd0f6b service swift] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:02:30.511686 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6dd02-7193-4862-8210-99c52ddd0f6b service swift] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:02:30.512988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6dd02-7193-4862-8210-99c52ddd0f6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:02:30.522320 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6dd02-7193-4862-8210-99c52ddd0f6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:02:30.523920 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6dd02-7193-4862-8210-99c52ddd0f6b service swift] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:02:30.531944 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1284/2611] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 21:02:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 21:02:40.373525 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=63 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:02:40.528697 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=38 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:02:42.098775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b49d99-6c4f-463f-b1ff-6429085415bc None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:02:42.099281 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b49d99-6c4f-463f-b1ff-6429085415bc None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:02:42.099423 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b49d99-6c4f-463f-b1ff-6429085415bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:02:42.119348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-25b49d99-6c4f-463f-b1ff-6429085415bc None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:02:42.120611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-25b49d99-6c4f-463f-b1ff-6429085415bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:02:42.128999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1328/2612] 10.222.0.60 () {58 vars in 1009 bytes} [Thu Jul 21 21:02:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:02:42.142253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6dd02-7193-4862-8210-99c52ddd0f6b service swift] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:02:42.152892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6dd02-7193-4862-8210-99c52ddd0f6b service swift] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:02:42.158333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a8bde-8d6b-4617-ab9a-d20120a6ee52 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:02:42.158988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a8bde-8d6b-4617-ab9a-d20120a6ee52 service swift] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:02:42.159325 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a8bde-8d6b-4617-ab9a-d20120a6ee52 service swift] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:02:42.159608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a8bde-8d6b-4617-ab9a-d20120a6ee52 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:02:42.170213 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a8bde-8d6b-4617-ab9a-d20120a6ee52 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:02:42.171844 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a8bde-8d6b-4617-ab9a-d20120a6ee52 service swift] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:02:42.181614 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1285/2613] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 21:02:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 21:02:44.718378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b49d99-6c4f-463f-b1ff-6429085415bc None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:02:44.719928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-25b49d99-6c4f-463f-b1ff-6429085415bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:02:44.736197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b49d99-6c4f-463f-b1ff-6429085415bc None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:02:44.742968 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bec75-48bc-43da-a638-02ef9446a315 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:02:44.743645 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bec75-48bc-43da-a638-02ef9446a315 service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:02:44.743962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bec75-48bc-43da-a638-02ef9446a315 service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:02:44.744237 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bec75-48bc-43da-a638-02ef9446a315 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:02:44.746065 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fc5bec75-48bc-43da-a638-02ef9446a315 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:02:44.866331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bec75-48bc-43da-a638-02ef9446a315 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:02:44.868388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bec75-48bc-43da-a638-02ef9446a315 service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:02:44.889829 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1329/2614] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:02:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:02:48.518370 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a465804-e5a1-4074-ab96-be2a77a20e34 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:02:48.519181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a465804-e5a1-4074-ab96-be2a77a20e34 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:02:48.519679 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a465804-e5a1-4074-ab96-be2a77a20e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:02:48.543198 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-6a465804-e5a1-4074-ab96-be2a77a20e34 None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:02:48.544899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6a465804-e5a1-4074-ab96-be2a77a20e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:02:48.553445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1286/2615] 10.222.0.60 () {58 vars in 1009 bytes} [Thu Jul 21 21:02:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:02:48.566882 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bec75-48bc-43da-a638-02ef9446a315 service nova] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:02:48.581677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bec75-48bc-43da-a638-02ef9446a315 service nova] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:02:48.587603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad809c9-bb28-40a7-9a7e-f52595f76422 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:02:48.588392 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad809c9-bb28-40a7-9a7e-f52595f76422 service swift] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:02:48.588751 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad809c9-bb28-40a7-9a7e-f52595f76422 service swift] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:02:48.589104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad809c9-bb28-40a7-9a7e-f52595f76422 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:02:48.599480 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad809c9-bb28-40a7-9a7e-f52595f76422 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:02:48.601585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad809c9-bb28-40a7-9a7e-f52595f76422 service swift] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:02:48.611962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1330/2616] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 21:02:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 21:02:58.107187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d045d1-ed54-470d-a74d-07e7a4d6a233 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:02:58.107769 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d045d1-ed54-470d-a74d-07e7a4d6a233 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:02:58.107948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d045d1-ed54-470d-a74d-07e7a4d6a233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:02:58.159816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c3d045d1-ed54-470d-a74d-07e7a4d6a233 None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:02:58.161020 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c3d045d1-ed54-470d-a74d-07e7a4d6a233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:02:58.180497 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1287/2617] 10.222.0.60 () {58 vars in 967 bytes} [Thu Jul 21 21:02:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:02:58.192828 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad809c9-bb28-40a7-9a7e-f52595f76422 service swift] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:02:58.215559 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad809c9-bb28-40a7-9a7e-f52595f76422 service swift] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:02:58.220970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c277be-5eba-4bc4-afb7-77fef4f7a6a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:02:58.221649 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c277be-5eba-4bc4-afb7-77fef4f7a6a0 service swift] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:02:58.221935 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c277be-5eba-4bc4-afb7-77fef4f7a6a0 service swift] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:02:58.222211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c277be-5eba-4bc4-afb7-77fef4f7a6a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:02:58.231982 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c277be-5eba-4bc4-afb7-77fef4f7a6a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:02:58.233636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c277be-5eba-4bc4-afb7-77fef4f7a6a0 service swift] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:02:58.242928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1331/2618] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 21:02:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 21:03:00.546379 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d045d1-ed54-470d-a74d-07e7a4d6a233 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:00.559619 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d045d1-ed54-470d-a74d-07e7a4d6a233 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:00.564174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35e10f-8783-48fe-ad90-ffbcf6b2918a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:00.565076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35e10f-8783-48fe-ad90-ffbcf6b2918a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:00.565357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35e10f-8783-48fe-ad90-ffbcf6b2918a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:00.565630 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35e10f-8783-48fe-ad90-ffbcf6b2918a admin admin] PATH_INFO: `/v3/users/356d7835faeb4ea09cf7580f5f467473` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:00.566825 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35e10f-8783-48fe-ad90-ffbcf6b2918a admin admin] RBAC: Authorizing `identity:delete_user(user_id=356d7835faeb4ea09cf7580f5f467473)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:00.569569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35e10f-8783-48fe-ad90-ffbcf6b2918a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:00.614608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9d35e10f-8783-48fe-ad90-ffbcf6b2918a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '356d7835faeb4ea09cf7580f5f467473'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:03:00.622487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9d35e10f-8783-48fe-ad90-ffbcf6b2918a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '356d7835faeb4ea09cf7580f5f467473'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:03:00.623103 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9d35e10f-8783-48fe-ad90-ffbcf6b2918a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '356d7835faeb4ea09cf7580f5f467473'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:03:00.630499 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9d35e10f-8783-48fe-ad90-ffbcf6b2918a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '356d7835faeb4ea09cf7580f5f467473'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:03:00.631894 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9d35e10f-8783-48fe-ad90-ffbcf6b2918a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '356d7835faeb4ea09cf7580f5f467473'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:03:00.649637 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1288/2619] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:03:00 2022] DELETE /identity/v3/users/356d7835faeb4ea09cf7580f5f467473 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:03:00.832044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c277be-5eba-4bc4-afb7-77fef4f7a6a0 service swift] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:00.848333 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c277be-5eba-4bc4-afb7-77fef4f7a6a0 service swift] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:00.905876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34189fc-e515-4a20-94ec-d6b697fe6298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:00.906658 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34189fc-e515-4a20-94ec-d6b697fe6298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:00.906941 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34189fc-e515-4a20-94ec-d6b697fe6298 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:00.907217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34189fc-e515-4a20-94ec-d6b697fe6298 admin admin] PATH_INFO: `/v3/projects/99bf599415154f1eb8ebdf982099e635` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:00.908451 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34189fc-e515-4a20-94ec-d6b697fe6298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99bf599415154f1eb8ebdf982099e635)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:00.910345 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34189fc-e515-4a20-94ec-d6b697fe6298 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:00.944400 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:03:00.960707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d34189fc-e515-4a20-94ec-d6b697fe6298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99bf599415154f1eb8ebdf982099e635', 'request_id': 'req-d34189fc-e515-4a20-94ec-d6b697fe6298'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:03:00.966793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d34189fc-e515-4a20-94ec-d6b697fe6298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99bf599415154f1eb8ebdf982099e635', 'request_id': 'req-d34189fc-e515-4a20-94ec-d6b697fe6298'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:03:00.972300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d34189fc-e515-4a20-94ec-d6b697fe6298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99bf599415154f1eb8ebdf982099e635', 'request_id': 'req-d34189fc-e515-4a20-94ec-d6b697fe6298'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:03:00.984964 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d34189fc-e515-4a20-94ec-d6b697fe6298 admin admin] The token cache is being invalidate because project 99bf599415154f1eb8ebdf982099e635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:03:00.985244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d34189fc-e515-4a20-94ec-d6b697fe6298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:03:00.985865 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d34189fc-e515-4a20-94ec-d6b697fe6298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:03:00.987425 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1332/2620] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:03:00 2022] DELETE /identity/v3/projects/99bf599415154f1eb8ebdf982099e635 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:03:00.993892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432ddc5-9605-43f3-b272-630f04151060 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:00.994189 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432ddc5-9605-43f3-b272-630f04151060 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:00.994469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432ddc5-9605-43f3-b272-630f04151060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:01.014701 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-7432ddc5-9605-43f3-b272-630f04151060 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:03:01.015956 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7432ddc5-9605-43f3-b272-630f04151060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:01.079347 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1289/2621] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:03:01.087244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed82456-ea61-40de-8522-b76de5214491 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:01.087559 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed82456-ea61-40de-8522-b76de5214491 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:01.087873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed82456-ea61-40de-8522-b76de5214491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:01.106085 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-5ed82456-ea61-40de-8522-b76de5214491 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:03:01.107275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-5ed82456-ea61-40de-8522-b76de5214491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:01.115786 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1333/2622] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:03:01.121856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432ddc5-9605-43f3-b272-630f04151060 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:01.132759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432ddc5-9605-43f3-b272-630f04151060 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:01.137368 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bd1ca-a191-45bd-9eba-86c530a7420e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:01.138021 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bd1ca-a191-45bd-9eba-86c530a7420e admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:01.138304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bd1ca-a191-45bd-9eba-86c530a7420e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:01.138578 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bd1ca-a191-45bd-9eba-86c530a7420e admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:01.139303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bd1ca-a191-45bd-9eba-86c530a7420e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:03:01.139608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bd1ca-a191-45bd-9eba-86c530a7420e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:01.141237 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bd1ca-a191-45bd-9eba-86c530a7420e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:01.150669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:03:01.155045 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1290/2623] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:03:01.159207 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed82456-ea61-40de-8522-b76de5214491 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:01.169321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed82456-ea61-40de-8522-b76de5214491 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:01.175706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5daf82b-307f-4159-9458-5e29f7056c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:01.176549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daf82b-307f-4159-9458-5e29f7056c09 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:01.176845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daf82b-307f-4159-9458-5e29f7056c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:01.177128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daf82b-307f-4159-9458-5e29f7056c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:01.178376 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5daf82b-307f-4159-9458-5e29f7056c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:01.180228 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5daf82b-307f-4159-9458-5e29f7056c09 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:01.220067 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1334/2624] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:03:01 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:03:01.224210 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bd1ca-a191-45bd-9eba-86c530a7420e admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:01.234770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bd1ca-a191-45bd-9eba-86c530a7420e admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:01.289458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83764805-8759-4deb-a2cc-3ec5287038c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:01.290122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83764805-8759-4deb-a2cc-3ec5287038c1 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:01.290402 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83764805-8759-4deb-a2cc-3ec5287038c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:01.290675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83764805-8759-4deb-a2cc-3ec5287038c1 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:01.291371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83764805-8759-4deb-a2cc-3ec5287038c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:01.293295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83764805-8759-4deb-a2cc-3ec5287038c1 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:01.312761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1291/2625] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:03:01 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:03:01.316899 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5daf82b-307f-4159-9458-5e29f7056c09 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:01.327514 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5daf82b-307f-4159-9458-5e29f7056c09 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:01.332841 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98c850-9f7c-40a2-ab82-195aad364397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:01.333546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98c850-9f7c-40a2-ab82-195aad364397 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:01.334348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98c850-9f7c-40a2-ab82-195aad364397 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:01.334693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98c850-9f7c-40a2-ab82-195aad364397 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:01.335592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98c850-9f7c-40a2-ab82-195aad364397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:01.338341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98c850-9f7c-40a2-ab82-195aad364397 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:01.348819 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1335/2626] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:03:01.353973 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83764805-8759-4deb-a2cc-3ec5287038c1 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:01.364023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83764805-8759-4deb-a2cc-3ec5287038c1 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:01.368944 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa25025-2d44-4ca9-b783-c915c6a4677f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:01.369605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa25025-2d44-4ca9-b783-c915c6a4677f admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:01.369887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa25025-2d44-4ca9-b783-c915c6a4677f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:01.370163 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa25025-2d44-4ca9-b783-c915c6a4677f admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:01.370894 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa25025-2d44-4ca9-b783-c915c6a4677f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:01.372512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa25025-2d44-4ca9-b783-c915c6a4677f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:01.382441 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1292/2627] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:03:01.386601 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98c850-9f7c-40a2-ab82-195aad364397 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:01.397369 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98c850-9f7c-40a2-ab82-195aad364397 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:01.402396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88d335-f8b5-4fe2-a527-1ae1e46e02ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:01.403272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88d335-f8b5-4fe2-a527-1ae1e46e02ef admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:01.403551 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88d335-f8b5-4fe2-a527-1ae1e46e02ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:01.403862 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88d335-f8b5-4fe2-a527-1ae1e46e02ef admin admin] PATH_INFO: `/v3/projects/6a26de2015bb4626ba4093aac7a3d085/users/084923b3deae41cd89c3935aed11a566/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:01.419521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88d335-f8b5-4fe2-a527-1ae1e46e02ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a26de2015bb4626ba4093aac7a3d085, user_id=084923b3deae41cd89c3935aed11a566, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:01.422124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88d335-f8b5-4fe2-a527-1ae1e46e02ef admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:01.435922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1336/2628] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:03:01 2022] PUT /identity/v3/projects/6a26de2015bb4626ba4093aac7a3d085/users/084923b3deae41cd89c3935aed11a566/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:03:01.441463 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d74d8-9d20-46fa-8b3a-1a72a1790c35 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:01.441932 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d74d8-9d20-46fa-8b3a-1a72a1790c35 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:01.442306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d74d8-9d20-46fa-8b3a-1a72a1790c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:01.461545 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-434d74d8-9d20-46fa-8b3a-1a72a1790c35 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:03:01.462763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-434d74d8-9d20-46fa-8b3a-1a72a1790c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:01.522953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1293/2629] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:03:01.533868 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88d335-f8b5-4fe2-a527-1ae1e46e02ef admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:01.535648 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8e88d335-f8b5-4fe2-a527-1ae1e46e02ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:01.593995 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88d335-f8b5-4fe2-a527-1ae1e46e02ef admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:01.597732 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54e153-26b3-45df-a046-b8095f7328b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:01.598376 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54e153-26b3-45df-a046-b8095f7328b1 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:01.598662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54e153-26b3-45df-a046-b8095f7328b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:01.598938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54e153-26b3-45df-a046-b8095f7328b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:01.608499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54e153-26b3-45df-a046-b8095f7328b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:01.609922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54e153-26b3-45df-a046-b8095f7328b1 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:01.620243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1337/2630] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:03:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 21:03:05.692626 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847a430-d9f0-40d3-bd9a-2dbbf0e3a407 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:05.693186 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847a430-d9f0-40d3-bd9a-2dbbf0e3a407 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:05.693284 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847a430-d9f0-40d3-bd9a-2dbbf0e3a407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:05.721551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c847a430-d9f0-40d3-bd9a-2dbbf0e3a407 None None] MFA Rules not processed for user `084923b3deae41cd89c3935aed11a566`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:03:05.722678 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c847a430-d9f0-40d3-bd9a-2dbbf0e3a407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:05.796887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1294/2631] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:03:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:03:05.801428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54e153-26b3-45df-a046-b8095f7328b1 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:05.811429 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54e153-26b3-45df-a046-b8095f7328b1 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:05.816094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa897f-3ca2-438c-8195-2623dc5eadeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:05.816788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa897f-3ca2-438c-8195-2623dc5eadeb admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:05.817081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa897f-3ca2-438c-8195-2623dc5eadeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:05.817359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa897f-3ca2-438c-8195-2623dc5eadeb admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:05.818084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa897f-3ca2-438c-8195-2623dc5eadeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:05.820142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa897f-3ca2-438c-8195-2623dc5eadeb admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:05.851750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1338/2632] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:03:05 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:03:05.855872 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847a430-d9f0-40d3-bd9a-2dbbf0e3a407 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:05.874673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847a430-d9f0-40d3-bd9a-2dbbf0e3a407 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:05.906649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:03:05.971966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab368456-0eab-4db7-b4fc-24f8bffff88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:05.972655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab368456-0eab-4db7-b4fc-24f8bffff88b admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:05.972940 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab368456-0eab-4db7-b4fc-24f8bffff88b admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:05.973223 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab368456-0eab-4db7-b4fc-24f8bffff88b admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:05.973922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab368456-0eab-4db7-b4fc-24f8bffff88b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:05.977051 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab368456-0eab-4db7-b4fc-24f8bffff88b admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:05.999886 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1295/2633] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:03:05 2022] POST /identity/v3/users => generated 298 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:03:06.003654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa897f-3ca2-438c-8195-2623dc5eadeb admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:06.023824 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa897f-3ca2-438c-8195-2623dc5eadeb admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:06.029615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d594c-1573-47cf-a7f0-cfd53bc30e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:06.030465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d594c-1573-47cf-a7f0-cfd53bc30e15 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:06.030820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d594c-1573-47cf-a7f0-cfd53bc30e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:06.031197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d594c-1573-47cf-a7f0-cfd53bc30e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:06.032538 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d594c-1573-47cf-a7f0-cfd53bc30e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:06.034880 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d594c-1573-47cf-a7f0-cfd53bc30e15 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:06.048716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1339/2634] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:03:06.053598 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab368456-0eab-4db7-b4fc-24f8bffff88b admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:06.065813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab368456-0eab-4db7-b4fc-24f8bffff88b admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:06.071394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9842f-19fc-4fb5-8181-2ea50661c28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:06.072345 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9842f-19fc-4fb5-8181-2ea50661c28c admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:06.072636 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9842f-19fc-4fb5-8181-2ea50661c28c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:06.072914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9842f-19fc-4fb5-8181-2ea50661c28c admin admin] PATH_INFO: `/v3/projects/ceb3e235e0a3417da5eec6a6fae8b102/users/cfe2160787d644fd85fca7ba4d7bfff4/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:06.093062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9842f-19fc-4fb5-8181-2ea50661c28c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb3e235e0a3417da5eec6a6fae8b102, user_id=cfe2160787d644fd85fca7ba4d7bfff4, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:06.096116 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9842f-19fc-4fb5-8181-2ea50661c28c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:06.113795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1296/2635] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:03:06 2022] PUT /identity/v3/projects/ceb3e235e0a3417da5eec6a6fae8b102/users/cfe2160787d644fd85fca7ba4d7bfff4/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:03:06.119321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d594c-1573-47cf-a7f0-cfd53bc30e15 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:06.137128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d594c-1573-47cf-a7f0-cfd53bc30e15 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:06.152441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:03:06.202539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2d08b-cab3-4194-9d73-7efc06fafa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:06.203203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2d08b-cab3-4194-9d73-7efc06fafa3c admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:06.203452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2d08b-cab3-4194-9d73-7efc06fafa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:06.203743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2d08b-cab3-4194-9d73-7efc06fafa3c admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:06.204603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2d08b-cab3-4194-9d73-7efc06fafa3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:06.206187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2d08b-cab3-4194-9d73-7efc06fafa3c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:06.221396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1340/2636] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:03:06.223227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9842f-19fc-4fb5-8181-2ea50661c28c admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:06.234930 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9842f-19fc-4fb5-8181-2ea50661c28c admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:06.239784 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ce75c-6f8d-49f4-9294-565a596fe037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:06.240805 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ce75c-6f8d-49f4-9294-565a596fe037 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:06.241383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ce75c-6f8d-49f4-9294-565a596fe037 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:06.241662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ce75c-6f8d-49f4-9294-565a596fe037 admin admin] PATH_INFO: `/v3/projects/ceb3e235e0a3417da5eec6a6fae8b102/users/cfe2160787d644fd85fca7ba4d7bfff4/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:06.243816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ce75c-6f8d-49f4-9294-565a596fe037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb3e235e0a3417da5eec6a6fae8b102, user_id=cfe2160787d644fd85fca7ba4d7bfff4, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:06.246480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ce75c-6f8d-49f4-9294-565a596fe037 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:06.259514 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1297/2637] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:03:06 2022] PUT /identity/v3/projects/ceb3e235e0a3417da5eec6a6fae8b102/users/cfe2160787d644fd85fca7ba4d7bfff4/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:03:10.439656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c232e1-3bf7-4778-bad7-b193a4e2281e None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:10.440194 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c232e1-3bf7-4778-bad7-b193a4e2281e None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:10.440297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c232e1-3bf7-4778-bad7-b193a4e2281e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:10.471148 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-a1c232e1-3bf7-4778-bad7-b193a4e2281e None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:03:10.472689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a1c232e1-3bf7-4778-bad7-b193a4e2281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:10.562003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1341/2638] 10.222.0.60 () {58 vars in 967 bytes} [Thu Jul 21 21:03:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:03:10.563070 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ce75c-6f8d-49f4-9294-565a596fe037 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:10.565730 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-fb2ce75c-6f8d-49f4-9294-565a596fe037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:10.731399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ce75c-6f8d-49f4-9294-565a596fe037 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:10.739529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f00ea-4fbf-4f17-9466-03a3b6cf497b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:10.741608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f00ea-4fbf-4f17-9466-03a3b6cf497b service swift] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:10.742224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f00ea-4fbf-4f17-9466-03a3b6cf497b service swift] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:10.742728 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f00ea-4fbf-4f17-9466-03a3b6cf497b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:10.753031 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f00ea-4fbf-4f17-9466-03a3b6cf497b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:10.754775 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f00ea-4fbf-4f17-9466-03a3b6cf497b service swift] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:10.783793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1298/2639] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 21:03:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 21:03:11.042993 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec3b9e-f499-4724-99bd-0f1a59d4257b None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:11.043289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec3b9e-f499-4724-99bd-0f1a59d4257b None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:11.043707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec3b9e-f499-4724-99bd-0f1a59d4257b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:11.077142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e9ec3b9e-f499-4724-99bd-0f1a59d4257b None None] MFA Rules not processed for user `cfe2160787d644fd85fca7ba4d7bfff4`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:03:11.078855 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e9ec3b9e-f499-4724-99bd-0f1a59d4257b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:11.167448 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1342/2640] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:03:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:03:11.189152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f00ea-4fbf-4f17-9466-03a3b6cf497b service swift] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:11.190655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-134f00ea-4fbf-4f17-9466-03a3b6cf497b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:11.253476 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:03:11.273412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f00ea-4fbf-4f17-9466-03a3b6cf497b service swift] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:11.277133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8bab7-8d90-4983-8a92-80689ab81568 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d596de177e24ca5a254bda6644a07da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:11.277792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8bab7-8d90-4983-8a92-80689ab81568 service glance] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:11.278078 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8bab7-8d90-4983-8a92-80689ab81568 service glance] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:11.278355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8bab7-8d90-4983-8a92-80689ab81568 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:11.337845 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8bab7-8d90-4983-8a92-80689ab81568 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:11.339282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8bab7-8d90-4983-8a92-80689ab81568 service glance] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:11.350367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1299/2641] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:03:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:03:11.479190 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b3ebb-041f-4e27-9b5c-0cc17f38ca17 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:11.480095 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b3ebb-041f-4e27-9b5c-0cc17f38ca17 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:11.480377 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b3ebb-041f-4e27-9b5c-0cc17f38ca17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:11.502098 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-8d4b3ebb-041f-4e27-9b5c-0cc17f38ca17 None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:03:11.503239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8d4b3ebb-041f-4e27-9b5c-0cc17f38ca17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:11.512598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1343/2642] 10.222.0.60 () {58 vars in 1009 bytes} [Thu Jul 21 21:03:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:03:11.528992 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8bab7-8d90-4983-8a92-80689ab81568 service glance] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:11.545075 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8bab7-8d90-4983-8a92-80689ab81568 service glance] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:11.550239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968eeac6-bf20-43da-9987-72c06c8117c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:11.550921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eeac6-bf20-43da-9987-72c06c8117c7 service swift] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:11.551202 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eeac6-bf20-43da-9987-72c06c8117c7 service swift] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:11.551477 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eeac6-bf20-43da-9987-72c06c8117c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:11.561255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968eeac6-bf20-43da-9987-72c06c8117c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:11.562867 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968eeac6-bf20-43da-9987-72c06c8117c7 service swift] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:11.571848 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1300/2643] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 21:03:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 21:03:20.323359 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52fce1-2b26-421a-b005-4b66b5eb7494 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:20.323890 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52fce1-2b26-421a-b005-4b66b5eb7494 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:20.324011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52fce1-2b26-421a-b005-4b66b5eb7494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:20.341021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-de52fce1-2b26-421a-b005-4b66b5eb7494 None None] MFA Rules not processed for user `084923b3deae41cd89c3935aed11a566`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:03:20.342236 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-de52fce1-2b26-421a-b005-4b66b5eb7494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:20.349596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1344/2644] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:03:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:03:20.917553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968eeac6-bf20-43da-9987-72c06c8117c7 service swift] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:20.919405 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-968eeac6-bf20-43da-9987-72c06c8117c7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:20.994796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968eeac6-bf20-43da-9987-72c06c8117c7 service swift] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:20.999910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850c78c-2da1-4f84-925b-3fad9488108d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:21.000556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850c78c-2da1-4f84-925b-3fad9488108d service placement] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:21.000873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850c78c-2da1-4f84-925b-3fad9488108d service placement] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:21.001152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850c78c-2da1-4f84-925b-3fad9488108d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:21.003058 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c850c78c-2da1-4f84-925b-3fad9488108d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:21.015206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850c78c-2da1-4f84-925b-3fad9488108d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:21.016840 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850c78c-2da1-4f84-925b-3fad9488108d service placement] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:21.027832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1301/2645] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:03:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:03:22.374313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5190e-88a0-47b3-88ac-93b0694c2621 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:22.374789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5190e-88a0-47b3-88ac-93b0694c2621 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:22.374900 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5190e-88a0-47b3-88ac-93b0694c2621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:22.393128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-81f5190e-88a0-47b3-88ac-93b0694c2621 None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:03:22.394476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-81f5190e-88a0-47b3-88ac-93b0694c2621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:22.403925 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1345/2646] 10.222.0.60 () {58 vars in 1009 bytes} [Thu Jul 21 21:03:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 21:03:22.417093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850c78c-2da1-4f84-925b-3fad9488108d service placement] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:22.428777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850c78c-2da1-4f84-925b-3fad9488108d service placement] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:22.433913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ce7b2-44cf-4fde-8ea9-be7af9e4b985 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:22.434643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ce7b2-44cf-4fde-8ea9-be7af9e4b985 service swift] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:22.434927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ce7b2-44cf-4fde-8ea9-be7af9e4b985 service swift] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:22.435198 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ce7b2-44cf-4fde-8ea9-be7af9e4b985 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:22.443439 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ce7b2-44cf-4fde-8ea9-be7af9e4b985 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:22.445616 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ce7b2-44cf-4fde-8ea9-be7af9e4b985 service swift] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:22.454095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1302/2647] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 21:03:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 21:03:32.387759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=90 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:03:32.450529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=74 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:03:36.957248 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5190e-88a0-47b3-88ac-93b0694c2621 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:36.960104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ce7b2-44cf-4fde-8ea9-be7af9e4b985 service swift] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:37.012330 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5190e-88a0-47b3-88ac-93b0694c2621 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:37.015926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18b682-a267-403a-972f-3bf0e7da572f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:37.016079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ce7b2-44cf-4fde-8ea9-be7af9e4b985 service swift] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:37.016578 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18b682-a267-403a-972f-3bf0e7da572f service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:37.016858 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18b682-a267-403a-972f-3bf0e7da572f service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:37.017132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18b682-a267-403a-972f-3bf0e7da572f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:37.018887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-3b18b682-a267-403a-972f-3bf0e7da572f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:37.019766 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3c8bc-183e-4671-ace2-2a3b9e1695d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:37.020419 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3c8bc-183e-4671-ace2-2a3b9e1695d5 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:37.020705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3c8bc-183e-4671-ace2-2a3b9e1695d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:37.021017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3c8bc-183e-4671-ace2-2a3b9e1695d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:37.022758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-06b3c8bc-183e-4671-ace2-2a3b9e1695d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:37.029487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18b682-a267-403a-972f-3bf0e7da572f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:37.030955 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18b682-a267-403a-972f-3bf0e7da572f service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:37.034564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3c8bc-183e-4671-ace2-2a3b9e1695d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:37.036006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3c8bc-183e-4671-ace2-2a3b9e1695d5 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:37.039348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1346/2648] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:03:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:03:37.044137 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1303/2649] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:03:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 21:03:44.517443 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9d8ae-9152-421d-bbf1-ff412a7e37ca None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:44.517963 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9d8ae-9152-421d-bbf1-ff412a7e37ca None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:44.518081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9d8ae-9152-421d-bbf1-ff412a7e37ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:44.535116 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-6bd9d8ae-9152-421d-bbf1-ff412a7e37ca None None] MFA Rules not processed for user `2b3617b8bcac4227b1295e294eec664a`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:03:44.536339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6bd9d8ae-9152-421d-bbf1-ff412a7e37ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:44.600009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1347/2650] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:03:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:03:47.040898 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=26 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:03:54.592382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=51 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:03:59.819280 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3c8bc-183e-4671-ace2-2a3b9e1695d5 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:03:59.820727 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-06b3c8bc-183e-4671-ace2-2a3b9e1695d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:03:59.831003 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3c8bc-183e-4671-ace2-2a3b9e1695d5 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:03:59.834576 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce5828-c571-4914-9c39-bfab1d52320d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:03:59.835206 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce5828-c571-4914-9c39-bfab1d52320d service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:03:59.835485 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce5828-c571-4914-9c39-bfab1d52320d service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:03:59.835799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce5828-c571-4914-9c39-bfab1d52320d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:03:59.845199 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce5828-c571-4914-9c39-bfab1d52320d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:03:59.846552 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce5828-c571-4914-9c39-bfab1d52320d service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:03:59.856672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1304/2651] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:03:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:04:00.837808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9d8ae-9152-421d-bbf1-ff412a7e37ca None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:04:00.839950 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6bd9d8ae-9152-421d-bbf1-ff412a7e37ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:04:00.924252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9d8ae-9152-421d-bbf1-ff412a7e37ca None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:04:00.929794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c4c37-caec-4c11-aae0-d547f73733fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:04:00.930489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c4c37-caec-4c11-aae0-d547f73733fd service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:00.930778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c4c37-caec-4c11-aae0-d547f73733fd service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:00.931058 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c4c37-caec-4c11-aae0-d547f73733fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:00.932884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-790c4c37-caec-4c11-aae0-d547f73733fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:04:00.945476 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c4c37-caec-4c11-aae0-d547f73733fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:04:00.947690 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c4c37-caec-4c11-aae0-d547f73733fd service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:04:00.959383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1348/2652] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:04:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:04:03.326434 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce5828-c571-4914-9c39-bfab1d52320d service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:04:03.346449 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce5828-c571-4914-9c39-bfab1d52320d service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:04:03.354604 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98581ae7-b998-4d08-8235-6f4fbbe5bf9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:04:03.355693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98581ae7-b998-4d08-8235-6f4fbbe5bf9e service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:03.356246 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98581ae7-b998-4d08-8235-6f4fbbe5bf9e service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:03.356863 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98581ae7-b998-4d08-8235-6f4fbbe5bf9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:03.358996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-98581ae7-b998-4d08-8235-6f4fbbe5bf9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:04:03.381157 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98581ae7-b998-4d08-8235-6f4fbbe5bf9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:04:03.382852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98581ae7-b998-4d08-8235-6f4fbbe5bf9e service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:04:03.428003 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1305/2653] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:04:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:04:04.956014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c4c37-caec-4c11-aae0-d547f73733fd service nova] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:04:04.965764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c4c37-caec-4c11-aae0-d547f73733fd service nova] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:04:04.969390 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7182960-5fa0-42c0-92f9-f0243f279090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:04:04.970028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7182960-5fa0-42c0-92f9-f0243f279090 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:04.970313 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7182960-5fa0-42c0-92f9-f0243f279090 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:04.970592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7182960-5fa0-42c0-92f9-f0243f279090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:04.972552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f7182960-5fa0-42c0-92f9-f0243f279090 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:04:05.037654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7182960-5fa0-42c0-92f9-f0243f279090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:04:05.039084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7182960-5fa0-42c0-92f9-f0243f279090 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:04:05.049440 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1349/2654] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:04:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:04:13.404757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=12 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:04:15.045045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=64 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:04:50.063359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98581ae7-b998-4d08-8235-6f4fbbe5bf9e service nova] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:04:50.073006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98581ae7-b998-4d08-8235-6f4fbbe5bf9e service nova] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:04:50.076816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73910e78-3e9b-4471-9739-41e3df0b36f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:04:50.077473 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73910e78-3e9b-4471-9739-41e3df0b36f6 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:50.077763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73910e78-3e9b-4471-9739-41e3df0b36f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:50.078040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73910e78-3e9b-4471-9739-41e3df0b36f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:50.079865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-73910e78-3e9b-4471-9739-41e3df0b36f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:04:50.142614 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73910e78-3e9b-4471-9739-41e3df0b36f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:04:50.144079 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73910e78-3e9b-4471-9739-41e3df0b36f6 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:04:50.154481 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1306/2655] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:04:53.498669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7182960-5fa0-42c0-92f9-f0243f279090 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:04:53.500244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f7182960-5fa0-42c0-92f9-f0243f279090 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:04:53.514701 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7182960-5fa0-42c0-92f9-f0243f279090 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:04:53.519247 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45740c81-34ca-463b-8e95-a184916e1472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:04:53.520206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45740c81-34ca-463b-8e95-a184916e1472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:53.520487 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45740c81-34ca-463b-8e95-a184916e1472 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:53.520760 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45740c81-34ca-463b-8e95-a184916e1472 admin admin] PATH_INFO: `/v3/users/3a0d900f08d54098870ebbfa281e2a65` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:53.521939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45740c81-34ca-463b-8e95-a184916e1472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a0d900f08d54098870ebbfa281e2a65)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:04:53.524502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45740c81-34ca-463b-8e95-a184916e1472 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:04:53.573104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45740c81-34ca-463b-8e95-a184916e1472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a0d900f08d54098870ebbfa281e2a65'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:04:53.573757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45740c81-34ca-463b-8e95-a184916e1472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a0d900f08d54098870ebbfa281e2a65'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:04:53.585623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45740c81-34ca-463b-8e95-a184916e1472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a0d900f08d54098870ebbfa281e2a65'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:04:53.595163 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45740c81-34ca-463b-8e95-a184916e1472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a0d900f08d54098870ebbfa281e2a65'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:04:53.602278 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45740c81-34ca-463b-8e95-a184916e1472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a0d900f08d54098870ebbfa281e2a65'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:04:53.608104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1350/2656] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:04:53 2022] DELETE /identity/v3/users/3a0d900f08d54098870ebbfa281e2a65 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:04:53.785590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73910e78-3e9b-4471-9739-41e3df0b36f6 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:04:53.795950 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73910e78-3e9b-4471-9739-41e3df0b36f6 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:04:53.853027 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79596b6c-4926-42db-89d1-0714d4ce4134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:04:53.853885 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79596b6c-4926-42db-89d1-0714d4ce4134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:53.854171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79596b6c-4926-42db-89d1-0714d4ce4134 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:53.854445 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79596b6c-4926-42db-89d1-0714d4ce4134 admin admin] PATH_INFO: `/v3/projects/c312164b3fdd40719fcdd94ed6bfd187` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:53.855921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79596b6c-4926-42db-89d1-0714d4ce4134 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c312164b3fdd40719fcdd94ed6bfd187)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:04:53.857823 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79596b6c-4926-42db-89d1-0714d4ce4134 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:04:53.905156 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-79596b6c-4926-42db-89d1-0714d4ce4134 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c312164b3fdd40719fcdd94ed6bfd187', 'request_id': 'req-79596b6c-4926-42db-89d1-0714d4ce4134'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:04:53.912202 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-79596b6c-4926-42db-89d1-0714d4ce4134 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c312164b3fdd40719fcdd94ed6bfd187', 'request_id': 'req-79596b6c-4926-42db-89d1-0714d4ce4134'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:04:53.918342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-79596b6c-4926-42db-89d1-0714d4ce4134 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c312164b3fdd40719fcdd94ed6bfd187', 'request_id': 'req-79596b6c-4926-42db-89d1-0714d4ce4134'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:04:53.929184 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-79596b6c-4926-42db-89d1-0714d4ce4134 admin admin] The token cache is being invalidate because project c312164b3fdd40719fcdd94ed6bfd187 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:04:53.930046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-79596b6c-4926-42db-89d1-0714d4ce4134 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:04:53.930683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-79596b6c-4926-42db-89d1-0714d4ce4134 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:04:53.932254 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1307/2657] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:04:53 2022] DELETE /identity/v3/projects/c312164b3fdd40719fcdd94ed6bfd187 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:04:53.937324 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa993841-bb4e-4f72-a204-2d30dd2bbed6 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:53.937623 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa993841-bb4e-4f72-a204-2d30dd2bbed6 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:53.937904 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa993841-bb4e-4f72-a204-2d30dd2bbed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:53.958639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-fa993841-bb4e-4f72-a204-2d30dd2bbed6 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:04:53.959957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fa993841-bb4e-4f72-a204-2d30dd2bbed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:04:54.021922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1351/2658] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:04:54.030584 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb4990-7f51-4ec8-9410-fac93b5c194f None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:54.030873 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb4990-7f51-4ec8-9410-fac93b5c194f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:54.031147 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb4990-7f51-4ec8-9410-fac93b5c194f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:54.049391 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-8deb4990-7f51-4ec8-9410-fac93b5c194f None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:04:54.050580 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8deb4990-7f51-4ec8-9410-fac93b5c194f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:04:54.059462 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1308/2659] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:04:54.063699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa993841-bb4e-4f72-a204-2d30dd2bbed6 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:04:54.074393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa993841-bb4e-4f72-a204-2d30dd2bbed6 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:04:54.078954 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbd762-d2b8-4519-a19f-7c870d2283a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:04:54.079621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbd762-d2b8-4519-a19f-7c870d2283a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:54.079938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbd762-d2b8-4519-a19f-7c870d2283a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:54.080219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbd762-d2b8-4519-a19f-7c870d2283a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:54.080954 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbd762-d2b8-4519-a19f-7c870d2283a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:04:54.081265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbd762-d2b8-4519-a19f-7c870d2283a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:04:54.082879 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbd762-d2b8-4519-a19f-7c870d2283a3 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:04:54.096126 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1352/2660] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:04:54.100575 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb4990-7f51-4ec8-9410-fac93b5c194f None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:04:54.111548 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb4990-7f51-4ec8-9410-fac93b5c194f None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:04:54.116632 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9b36c-bd8c-44e7-920e-284f9d338e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:04:54.117838 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9b36c-bd8c-44e7-920e-284f9d338e03 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:54.118222 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9b36c-bd8c-44e7-920e-284f9d338e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:54.118508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9b36c-bd8c-44e7-920e-284f9d338e03 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:54.119316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9b36c-bd8c-44e7-920e-284f9d338e03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:04:54.121606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9b36c-bd8c-44e7-920e-284f9d338e03 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:04:54.143813 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:04:54.158065 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1309/2661] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:04:54 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:04:54.162429 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbd762-d2b8-4519-a19f-7c870d2283a3 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:04:54.172506 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbd762-d2b8-4519-a19f-7c870d2283a3 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:04:54.228571 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f6865-fc96-4cc0-810d-d35088c20926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:04:54.229149 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f6865-fc96-4cc0-810d-d35088c20926 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:54.229437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f6865-fc96-4cc0-810d-d35088c20926 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:54.229718 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f6865-fc96-4cc0-810d-d35088c20926 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:54.230424 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f6865-fc96-4cc0-810d-d35088c20926 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:04:54.232383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f6865-fc96-4cc0-810d-d35088c20926 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:04:54.238984 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:04:54.253732 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1353/2662] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:04:54 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 21:04:54.258773 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9b36c-bd8c-44e7-920e-284f9d338e03 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:04:54.269877 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9b36c-bd8c-44e7-920e-284f9d338e03 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:04:54.275158 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17161f02-408f-4c37-a92d-e95f0145a2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:04:54.275880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17161f02-408f-4c37-a92d-e95f0145a2ee admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:54.276169 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17161f02-408f-4c37-a92d-e95f0145a2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:54.276453 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17161f02-408f-4c37-a92d-e95f0145a2ee admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:54.277182 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17161f02-408f-4c37-a92d-e95f0145a2ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:04:54.278787 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17161f02-408f-4c37-a92d-e95f0145a2ee admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:04:54.289346 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1310/2663] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:04:54.293100 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f6865-fc96-4cc0-810d-d35088c20926 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:04:54.302883 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f6865-fc96-4cc0-810d-d35088c20926 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:04:54.308054 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c0c10-e955-488d-a15a-5791c1d08a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:04:54.308696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c0c10-e955-488d-a15a-5791c1d08a80 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:54.308979 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c0c10-e955-488d-a15a-5791c1d08a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:54.309254 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c0c10-e955-488d-a15a-5791c1d08a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:54.309999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c0c10-e955-488d-a15a-5791c1d08a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:04:54.311575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c0c10-e955-488d-a15a-5791c1d08a80 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:04:54.322450 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1354/2664] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:04:54.326424 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17161f02-408f-4c37-a92d-e95f0145a2ee admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:04:54.336691 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17161f02-408f-4c37-a92d-e95f0145a2ee admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:04:54.341474 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7097075-0d4e-4dd1-bb93-3521fa2b6e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:04:54.342336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7097075-0d4e-4dd1-bb93-3521fa2b6e2c admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:54.342617 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7097075-0d4e-4dd1-bb93-3521fa2b6e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:54.342890 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7097075-0d4e-4dd1-bb93-3521fa2b6e2c admin admin] PATH_INFO: `/v3/projects/72d4aa110cf348049f72ae629879beea/users/12941ff7dc2c42b18dcc5acfa79275c2/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:54.357328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7097075-0d4e-4dd1-bb93-3521fa2b6e2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d4aa110cf348049f72ae629879beea, user_id=12941ff7dc2c42b18dcc5acfa79275c2, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:04:54.359967 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7097075-0d4e-4dd1-bb93-3521fa2b6e2c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:04:54.373772 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1311/2665] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:04:54 2022] PUT /identity/v3/projects/72d4aa110cf348049f72ae629879beea/users/12941ff7dc2c42b18dcc5acfa79275c2/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:04:54.385395 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4094a-3c67-4c16-879a-0f88e6c47365 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:54.385715 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4094a-3c67-4c16-879a-0f88e6c47365 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:54.386235 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4094a-3c67-4c16-879a-0f88e6c47365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:54.418512 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d4e4094a-3c67-4c16-879a-0f88e6c47365 None None] MFA Rules not processed for user `12941ff7dc2c42b18dcc5acfa79275c2`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:04:54.419769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d4e4094a-3c67-4c16-879a-0f88e6c47365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:04:54.487664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1355/2666] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:04:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:04:54.499835 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7097075-0d4e-4dd1-bb93-3521fa2b6e2c admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:04:54.501620 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d7097075-0d4e-4dd1-bb93-3521fa2b6e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:04:54.578188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7097075-0d4e-4dd1-bb93-3521fa2b6e2c admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:04:54.583947 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b5324-9c69-4f26-8b8a-22b8f0915dde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:04:54.584605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b5324-9c69-4f26-8b8a-22b8f0915dde service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:04:54.584918 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b5324-9c69-4f26-8b8a-22b8f0915dde service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:04:54.585199 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b5324-9c69-4f26-8b8a-22b8f0915dde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:04:54.593492 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b5324-9c69-4f26-8b8a-22b8f0915dde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:04:54.595084 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b5324-9c69-4f26-8b8a-22b8f0915dde service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:04:54.604481 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1312/2667] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:04:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:05:04.484781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:04.600110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=51,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:05.036452 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4094a-3c67-4c16-879a-0f88e6c47365 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:05.117062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4094a-3c67-4c16-879a-0f88e6c47365 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:05.122224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d9d9a-d079-4f1e-b094-f3ee5f6f4adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:05.123134 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d9d9a-d079-4f1e-b094-f3ee5f6f4adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:05.123415 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d9d9a-d079-4f1e-b094-f3ee5f6f4adb admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:05.123687 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d9d9a-d079-4f1e-b094-f3ee5f6f4adb admin admin] PATH_INFO: `/v3/users/12941ff7dc2c42b18dcc5acfa79275c2` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:05.125016 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d9d9a-d079-4f1e-b094-f3ee5f6f4adb admin admin] RBAC: Authorizing `identity:delete_user(user_id=12941ff7dc2c42b18dcc5acfa79275c2)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:05.126954 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d9d9a-d079-4f1e-b094-f3ee5f6f4adb admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:05.193351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-ce3d9d9a-d079-4f1e-b094-f3ee5f6f4adb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12941ff7dc2c42b18dcc5acfa79275c2'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:05.195264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-ce3d9d9a-d079-4f1e-b094-f3ee5f6f4adb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12941ff7dc2c42b18dcc5acfa79275c2'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:05.219572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-ce3d9d9a-d079-4f1e-b094-f3ee5f6f4adb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12941ff7dc2c42b18dcc5acfa79275c2'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:05.228516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-ce3d9d9a-d079-4f1e-b094-f3ee5f6f4adb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12941ff7dc2c42b18dcc5acfa79275c2'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:05.238106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-ce3d9d9a-d079-4f1e-b094-f3ee5f6f4adb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12941ff7dc2c42b18dcc5acfa79275c2'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:05.246842 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1356/2668] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:05:05 2022] DELETE /identity/v3/users/12941ff7dc2c42b18dcc5acfa79275c2 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:05.251306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee52148-6727-4a28-86af-ec0014265649 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:05.251605 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee52148-6727-4a28-86af-ec0014265649 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:05.251923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee52148-6727-4a28-86af-ec0014265649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:05.272937 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-7ee52148-6727-4a28-86af-ec0014265649 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:05.274424 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7ee52148-6727-4a28-86af-ec0014265649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:05.345117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1313/2669] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:05.357170 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d9d9a-d079-4f1e-b094-f3ee5f6f4adb admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:05.358791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ce3d9d9a-d079-4f1e-b094-f3ee5f6f4adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:05.428611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d9d9a-d079-4f1e-b094-f3ee5f6f4adb admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:05.434913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8d0bf-4938-487b-aa8a-7487329de1a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:05.436019 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8d0bf-4938-487b-aa8a-7487329de1a7 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:05.436297 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8d0bf-4938-487b-aa8a-7487329de1a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:05.436576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8d0bf-4938-487b-aa8a-7487329de1a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:05.448652 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8d0bf-4938-487b-aa8a-7487329de1a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:05.450260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8d0bf-4938-487b-aa8a-7487329de1a7 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:05.462214 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1357/2670] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:05:05.898240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee52148-6727-4a28-86af-ec0014265649 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:05.899791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7ee52148-6727-4a28-86af-ec0014265649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:05.912255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee52148-6727-4a28-86af-ec0014265649 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:05.918188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858dee07-e101-4b5b-8607-b73b06a0320f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:05.918935 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858dee07-e101-4b5b-8607-b73b06a0320f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:05.919213 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858dee07-e101-4b5b-8607-b73b06a0320f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:05.919485 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858dee07-e101-4b5b-8607-b73b06a0320f admin admin] PATH_INFO: `/v3/projects/72d4aa110cf348049f72ae629879beea` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:05.920708 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858dee07-e101-4b5b-8607-b73b06a0320f admin admin] RBAC: Authorizing `identity:delete_project(project_id=72d4aa110cf348049f72ae629879beea)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:05.922622 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858dee07-e101-4b5b-8607-b73b06a0320f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:05.974933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-858dee07-e101-4b5b-8607-b73b06a0320f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72d4aa110cf348049f72ae629879beea', 'request_id': 'req-858dee07-e101-4b5b-8607-b73b06a0320f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:05.982955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-858dee07-e101-4b5b-8607-b73b06a0320f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72d4aa110cf348049f72ae629879beea', 'request_id': 'req-858dee07-e101-4b5b-8607-b73b06a0320f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:05.992666 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-858dee07-e101-4b5b-8607-b73b06a0320f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72d4aa110cf348049f72ae629879beea', 'request_id': 'req-858dee07-e101-4b5b-8607-b73b06a0320f'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:06.004594 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-858dee07-e101-4b5b-8607-b73b06a0320f admin admin] The token cache is being invalidate because project 72d4aa110cf348049f72ae629879beea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:05:06.004883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-858dee07-e101-4b5b-8607-b73b06a0320f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:06.005556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-858dee07-e101-4b5b-8607-b73b06a0320f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:06.007158 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1314/2671] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:05:05 2022] DELETE /identity/v3/projects/72d4aa110cf348049f72ae629879beea => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:06.012922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ab753-83d6-46a0-84ae-a5be02f6a3fc None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:06.013401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ab753-83d6-46a0-84ae-a5be02f6a3fc None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:06.013515 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ab753-83d6-46a0-84ae-a5be02f6a3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:06.032044 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-611ab753-83d6-46a0-84ae-a5be02f6a3fc None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:06.033257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-611ab753-83d6-46a0-84ae-a5be02f6a3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:06.051103 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:06.100615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1358/2672] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 21:05:06.111255 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07661f6-c5ff-4b7c-81a5-ae2e05c90957 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:06.111553 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07661f6-c5ff-4b7c-81a5-ae2e05c90957 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:06.111915 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07661f6-c5ff-4b7c-81a5-ae2e05c90957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:06.136836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c07661f6-c5ff-4b7c-81a5-ae2e05c90957 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:06.138062 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c07661f6-c5ff-4b7c-81a5-ae2e05c90957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:06.146218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1315/2673] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:06.150772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ab753-83d6-46a0-84ae-a5be02f6a3fc None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:06.160529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ab753-83d6-46a0-84ae-a5be02f6a3fc None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:06.166772 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4721a48-293f-45c4-b1b1-83306c2738d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:06.167432 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4721a48-293f-45c4-b1b1-83306c2738d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:06.167764 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4721a48-293f-45c4-b1b1-83306c2738d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:06.168043 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4721a48-293f-45c4-b1b1-83306c2738d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:06.168762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4721a48-293f-45c4-b1b1-83306c2738d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:05:06.169064 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4721a48-293f-45c4-b1b1-83306c2738d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:06.170677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4721a48-293f-45c4-b1b1-83306c2738d1 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:06.184112 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1359/2674] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:05:06.188539 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07661f6-c5ff-4b7c-81a5-ae2e05c90957 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:06.198215 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07661f6-c5ff-4b7c-81a5-ae2e05c90957 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:06.203023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04752e89-b4f1-4acb-9e6d-3b6119518440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:06.203683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04752e89-b4f1-4acb-9e6d-3b6119518440 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:06.204000 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04752e89-b4f1-4acb-9e6d-3b6119518440 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:06.204281 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04752e89-b4f1-4acb-9e6d-3b6119518440 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:06.205019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04752e89-b4f1-4acb-9e6d-3b6119518440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:06.206914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04752e89-b4f1-4acb-9e6d-3b6119518440 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:06.241021 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1316/2675] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:05:06 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:05:06.242418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4721a48-293f-45c4-b1b1-83306c2738d1 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:06.253134 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4721a48-293f-45c4-b1b1-83306c2738d1 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:06.309208 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa6d8e-796d-4049-a71f-60f5d018525c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:06.309798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa6d8e-796d-4049-a71f-60f5d018525c admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:06.310085 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa6d8e-796d-4049-a71f-60f5d018525c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:06.310366 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa6d8e-796d-4049-a71f-60f5d018525c admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:06.311057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa6d8e-796d-4049-a71f-60f5d018525c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:06.313006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa6d8e-796d-4049-a71f-60f5d018525c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:06.332265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1360/2676] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:05:06 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 21:05:06.336885 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04752e89-b4f1-4acb-9e6d-3b6119518440 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:06.346949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04752e89-b4f1-4acb-9e6d-3b6119518440 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:06.351501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d5ca9-eec1-46ac-89aa-1a2760dd919a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:06.352909 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d5ca9-eec1-46ac-89aa-1a2760dd919a admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:06.353188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d5ca9-eec1-46ac-89aa-1a2760dd919a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:06.353461 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d5ca9-eec1-46ac-89aa-1a2760dd919a admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:06.354181 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d5ca9-eec1-46ac-89aa-1a2760dd919a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:06.355824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d5ca9-eec1-46ac-89aa-1a2760dd919a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:06.367854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1317/2677] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:06.371679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa6d8e-796d-4049-a71f-60f5d018525c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:06.381650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa6d8e-796d-4049-a71f-60f5d018525c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:06.386279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16275b-20e4-4192-b2a2-247c1f7fa259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:06.386926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16275b-20e4-4192-b2a2-247c1f7fa259 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:06.387209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16275b-20e4-4192-b2a2-247c1f7fa259 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:06.387573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16275b-20e4-4192-b2a2-247c1f7fa259 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:06.388331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16275b-20e4-4192-b2a2-247c1f7fa259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:06.389903 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16275b-20e4-4192-b2a2-247c1f7fa259 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:06.402803 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1361/2678] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:06.404514 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d5ca9-eec1-46ac-89aa-1a2760dd919a admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:06.415202 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d5ca9-eec1-46ac-89aa-1a2760dd919a admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:06.420027 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2753b5-acea-4632-9c69-2eed7c7b513a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:06.420903 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2753b5-acea-4632-9c69-2eed7c7b513a admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:06.421192 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2753b5-acea-4632-9c69-2eed7c7b513a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:06.421461 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2753b5-acea-4632-9c69-2eed7c7b513a admin admin] PATH_INFO: `/v3/projects/77c7cfe2e6954c3da403f754cfc9a52c/users/e515ec9a289542fea86506e43a3c5bc1/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:06.436272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2753b5-acea-4632-9c69-2eed7c7b513a admin admin] RBAC: Authorizing `identity:create_grant(project_id=77c7cfe2e6954c3da403f754cfc9a52c, user_id=e515ec9a289542fea86506e43a3c5bc1, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:06.438904 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2753b5-acea-4632-9c69-2eed7c7b513a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:06.451377 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1318/2679] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:05:06 2022] PUT /identity/v3/projects/77c7cfe2e6954c3da403f754cfc9a52c/users/e515ec9a289542fea86506e43a3c5bc1/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:06.461910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf286c-a464-4989-871f-f37e7985fefa None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:06.462275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf286c-a464-4989-871f-f37e7985fefa None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:06.462563 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf286c-a464-4989-871f-f37e7985fefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:06.491334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-14bf286c-a464-4989-871f-f37e7985fefa None None] MFA Rules not processed for user `e515ec9a289542fea86506e43a3c5bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:06.493664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-14bf286c-a464-4989-871f-f37e7985fefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:06.560002 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1362/2680] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:06.564516 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2753b5-acea-4632-9c69-2eed7c7b513a admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:06.587985 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:06.635143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2753b5-acea-4632-9c69-2eed7c7b513a admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:06.640558 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b01bb1-0f10-4b78-97c6-40f505a22938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:06.641336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b01bb1-0f10-4b78-97c6-40f505a22938 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:06.642069 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b01bb1-0f10-4b78-97c6-40f505a22938 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:06.642069 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b01bb1-0f10-4b78-97c6-40f505a22938 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:06.642765 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b01bb1-0f10-4b78-97c6-40f505a22938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:06.644733 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b01bb1-0f10-4b78-97c6-40f505a22938 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:06.674879 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1319/2681] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:05:06 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:05:06.676074 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf286c-a464-4989-871f-f37e7985fefa None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:06.687537 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf286c-a464-4989-871f-f37e7985fefa None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:06.700000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:06.745745 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17356891-36ca-4507-817c-5ea249d87bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:06.746413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17356891-36ca-4507-817c-5ea249d87bfc admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:06.746700 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17356891-36ca-4507-817c-5ea249d87bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:06.746980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17356891-36ca-4507-817c-5ea249d87bfc admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:06.747671 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17356891-36ca-4507-817c-5ea249d87bfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:06.749636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17356891-36ca-4507-817c-5ea249d87bfc admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:06.769321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1363/2682] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:05:06 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 21:05:06.773356 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b01bb1-0f10-4b78-97c6-40f505a22938 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:06.783728 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b01bb1-0f10-4b78-97c6-40f505a22938 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:06.788396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ebd9a-610b-4314-b6c5-4635db7dcfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:06.789059 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ebd9a-610b-4314-b6c5-4635db7dcfb2 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:06.790041 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ebd9a-610b-4314-b6c5-4635db7dcfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:06.790339 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ebd9a-610b-4314-b6c5-4635db7dcfb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:06.791071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ebd9a-610b-4314-b6c5-4635db7dcfb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:06.792705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ebd9a-610b-4314-b6c5-4635db7dcfb2 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:06.803106 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1320/2683] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:06.807630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17356891-36ca-4507-817c-5ea249d87bfc admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:06.817550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17356891-36ca-4507-817c-5ea249d87bfc admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:06.822708 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7ffd9-b019-4383-8aaf-eceaade6b689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:06.823348 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7ffd9-b019-4383-8aaf-eceaade6b689 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:06.823628 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7ffd9-b019-4383-8aaf-eceaade6b689 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:06.823936 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7ffd9-b019-4383-8aaf-eceaade6b689 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:06.824684 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7ffd9-b019-4383-8aaf-eceaade6b689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:06.826404 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7ffd9-b019-4383-8aaf-eceaade6b689 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:06.837524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1364/2684] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:06.841596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ebd9a-610b-4314-b6c5-4635db7dcfb2 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:06.852208 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ebd9a-610b-4314-b6c5-4635db7dcfb2 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:06.856933 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7befb0-c6e8-4072-9d58-41b11ef94a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:06.857807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7befb0-c6e8-4072-9d58-41b11ef94a65 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:06.858094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7befb0-c6e8-4072-9d58-41b11ef94a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:06.858394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7befb0-c6e8-4072-9d58-41b11ef94a65 admin admin] PATH_INFO: `/v3/projects/bd31eff7f1434830a1565c05eb1a42b8/users/b44a09f74ea64024b6f766a4841c44bc/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:06.873529 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7befb0-c6e8-4072-9d58-41b11ef94a65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd31eff7f1434830a1565c05eb1a42b8, user_id=b44a09f74ea64024b6f766a4841c44bc, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:06.876209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7befb0-c6e8-4072-9d58-41b11ef94a65 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:06.890359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1321/2685] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:05:06 2022] PUT /identity/v3/projects/bd31eff7f1434830a1565c05eb1a42b8/users/b44a09f74ea64024b6f766a4841c44bc/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:06.902380 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c65229-83dc-4343-a82e-1fc7311ee80c None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:06.902703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c65229-83dc-4343-a82e-1fc7311ee80c None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:06.902989 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c65229-83dc-4343-a82e-1fc7311ee80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:06.929643 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-76c65229-83dc-4343-a82e-1fc7311ee80c None None] MFA Rules not processed for user `b44a09f74ea64024b6f766a4841c44bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:06.930936 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-76c65229-83dc-4343-a82e-1fc7311ee80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:07.004174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1365/2686] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:07.009397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7befb0-c6e8-4072-9d58-41b11ef94a65 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:07.070510 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7befb0-c6e8-4072-9d58-41b11ef94a65 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:07.075750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222102e6-173b-473e-bac7-18a7ca3b6ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:07.076413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222102e6-173b-473e-bac7-18a7ca3b6ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:07.076701 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222102e6-173b-473e-bac7-18a7ca3b6ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:07.076986 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222102e6-173b-473e-bac7-18a7ca3b6ad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:07.077690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222102e6-173b-473e-bac7-18a7ca3b6ad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:07.079583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222102e6-173b-473e-bac7-18a7ca3b6ad1 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:07.105242 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1322/2687] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:05:07 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:05:07.109303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c65229-83dc-4343-a82e-1fc7311ee80c None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:07.120967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c65229-83dc-4343-a82e-1fc7311ee80c None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:07.179669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afffa63-e33e-4c36-89f8-bdc4a8bd25a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:07.180410 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afffa63-e33e-4c36-89f8-bdc4a8bd25a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:07.180703 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afffa63-e33e-4c36-89f8-bdc4a8bd25a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:07.180977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afffa63-e33e-4c36-89f8-bdc4a8bd25a9 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:07.181672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afffa63-e33e-4c36-89f8-bdc4a8bd25a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:07.183584 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afffa63-e33e-4c36-89f8-bdc4a8bd25a9 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:07.209189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1366/2688] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:05:07 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 21:05:07.211782 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222102e6-173b-473e-bac7-18a7ca3b6ad1 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:07.223163 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222102e6-173b-473e-bac7-18a7ca3b6ad1 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:07.228538 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38982521-3cf1-4d89-87cf-10681ff45ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:07.229209 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38982521-3cf1-4d89-87cf-10681ff45ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:07.229671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38982521-3cf1-4d89-87cf-10681ff45ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:07.229952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38982521-3cf1-4d89-87cf-10681ff45ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:07.230682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38982521-3cf1-4d89-87cf-10681ff45ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:07.232298 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38982521-3cf1-4d89-87cf-10681ff45ec4 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:07.243853 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1323/2689] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:07.255163 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afffa63-e33e-4c36-89f8-bdc4a8bd25a9 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:07.258654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afffa63-e33e-4c36-89f8-bdc4a8bd25a9 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:07.263333 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a076063-4e2e-445d-8efd-24f6d9a99f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:07.264236 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a076063-4e2e-445d-8efd-24f6d9a99f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:07.264517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a076063-4e2e-445d-8efd-24f6d9a99f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:07.264790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a076063-4e2e-445d-8efd-24f6d9a99f67 admin admin] PATH_INFO: `/v3/projects/700a5011aecf4d5b80a0176b1f4c9a01/users/e5a1a6768c1c4ac68d756c7dc2ba60ee/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:07.277916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:07.281292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a076063-4e2e-445d-8efd-24f6d9a99f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=700a5011aecf4d5b80a0176b1f4c9a01, user_id=e5a1a6768c1c4ac68d756c7dc2ba60ee, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:07.284580 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a076063-4e2e-445d-8efd-24f6d9a99f67 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:07.298029 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1367/2690] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:05:07 2022] PUT /identity/v3/projects/700a5011aecf4d5b80a0176b1f4c9a01/users/e5a1a6768c1c4ac68d756c7dc2ba60ee/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:05:07.302158 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38982521-3cf1-4d89-87cf-10681ff45ec4 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:07.312195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38982521-3cf1-4d89-87cf-10681ff45ec4 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:07.353864 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:07.370103 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d6bc5-a703-4941-8d8b-4fa64b7d2d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:07.370796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d6bc5-a703-4941-8d8b-4fa64b7d2d57 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:07.371080 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d6bc5-a703-4941-8d8b-4fa64b7d2d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:07.371357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d6bc5-a703-4941-8d8b-4fa64b7d2d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:07.372117 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d6bc5-a703-4941-8d8b-4fa64b7d2d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:07.373712 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d6bc5-a703-4941-8d8b-4fa64b7d2d57 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:07.384225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1324/2691] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 21:05:07.388607 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a076063-4e2e-445d-8efd-24f6d9a99f67 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:07.399383 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a076063-4e2e-445d-8efd-24f6d9a99f67 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:07.404559 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecc04c-425d-4b8e-95ae-e943af783c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:07.404973 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecc04c-425d-4b8e-95ae-e943af783c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:07.405259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecc04c-425d-4b8e-95ae-e943af783c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:07.405541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecc04c-425d-4b8e-95ae-e943af783c1d admin admin] PATH_INFO: `/v3/projects/700a5011aecf4d5b80a0176b1f4c9a01/users/e5a1a6768c1c4ac68d756c7dc2ba60ee/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:07.407548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecc04c-425d-4b8e-95ae-e943af783c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=700a5011aecf4d5b80a0176b1f4c9a01, user_id=e5a1a6768c1c4ac68d756c7dc2ba60ee, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:07.410232 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecc04c-425d-4b8e-95ae-e943af783c1d admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:07.421124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1368/2692] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:05:07 2022] PUT /identity/v3/projects/700a5011aecf4d5b80a0176b1f4c9a01/users/e5a1a6768c1c4ac68d756c7dc2ba60ee/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:07.431422 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c6103-78de-4abc-aacd-25167b2cfdb8 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:07.431735 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c6103-78de-4abc-aacd-25167b2cfdb8 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:07.432027 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c6103-78de-4abc-aacd-25167b2cfdb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:07.458892 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-e83c6103-78de-4abc-aacd-25167b2cfdb8 None None] MFA Rules not processed for user `e5a1a6768c1c4ac68d756c7dc2ba60ee`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:07.460134 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e83c6103-78de-4abc-aacd-25167b2cfdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:07.530252 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1325/2693] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 21:05:07.541579 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecc04c-425d-4b8e-95ae-e943af783c1d admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:07.543395 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-0eecc04c-425d-4b8e-95ae-e943af783c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:07.600304 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecc04c-425d-4b8e-95ae-e943af783c1d admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:07.604964 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8731493-fa98-4896-95b7-785e9c876486 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:07.605548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8731493-fa98-4896-95b7-785e9c876486 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:07.605834 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8731493-fa98-4896-95b7-785e9c876486 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:07.606113 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8731493-fa98-4896-95b7-785e9c876486 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:07.615575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8731493-fa98-4896-95b7-785e9c876486 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:07.617045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8731493-fa98-4896-95b7-785e9c876486 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:07.627685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1369/2694] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:05:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:05:08.133718 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c6103-78de-4abc-aacd-25167b2cfdb8 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:08.135737 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e83c6103-78de-4abc-aacd-25167b2cfdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:08.147591 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c6103-78de-4abc-aacd-25167b2cfdb8 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:08.151675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a562702-b2cf-4162-8bd2-e5c4a8e982b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:08.152340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a562702-b2cf-4162-8bd2-e5c4a8e982b3 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:08.152622 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a562702-b2cf-4162-8bd2-e5c4a8e982b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:08.152895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a562702-b2cf-4162-8bd2-e5c4a8e982b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:08.209453 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a562702-b2cf-4162-8bd2-e5c4a8e982b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:08.210880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a562702-b2cf-4162-8bd2-e5c4a8e982b3 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:08.221320 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1326/2695] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:05:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:05:09.574550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8731493-fa98-4896-95b7-785e9c876486 service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:09.576321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f8731493-fa98-4896-95b7-785e9c876486 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:09.637141 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8731493-fa98-4896-95b7-785e9c876486 service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:09.640833 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce8b6c-444f-401b-9254-01e2ee547b68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:09.641481 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce8b6c-444f-401b-9254-01e2ee547b68 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:09.641773 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce8b6c-444f-401b-9254-01e2ee547b68 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:09.642051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce8b6c-444f-401b-9254-01e2ee547b68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:09.643972 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9fce8b6c-444f-401b-9254-01e2ee547b68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:09.709627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce8b6c-444f-401b-9254-01e2ee547b68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:09.711496 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce8b6c-444f-401b-9254-01e2ee547b68 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:09.723127 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1370/2696] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:05:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:05:12.248741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a562702-b2cf-4162-8bd2-e5c4a8e982b3 service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:12.258485 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a562702-b2cf-4162-8bd2-e5c4a8e982b3 service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:12.263200 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349e4db-3717-42bf-995a-efde18019053 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:12.263935 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349e4db-3717-42bf-995a-efde18019053 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:12.264225 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349e4db-3717-42bf-995a-efde18019053 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:12.264508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349e4db-3717-42bf-995a-efde18019053 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:12.324753 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349e4db-3717-42bf-995a-efde18019053 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:12.326195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349e4db-3717-42bf-995a-efde18019053 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:12.335661 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1327/2697] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:05:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:05:12.960101 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce8b6c-444f-401b-9254-01e2ee547b68 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:12.962758 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9fce8b6c-444f-401b-9254-01e2ee547b68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:12.977722 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce8b6c-444f-401b-9254-01e2ee547b68 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:12.988812 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507e8e2-7118-4d3d-8132-9956d49be23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:12.988812 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507e8e2-7118-4d3d-8132-9956d49be23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:12.988812 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507e8e2-7118-4d3d-8132-9956d49be23b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:12.988812 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507e8e2-7118-4d3d-8132-9956d49be23b admin admin] PATH_INFO: `/v3/users/ab2bc1eb1eb845ec82b79adee04a51e7` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:13.000483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507e8e2-7118-4d3d-8132-9956d49be23b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab2bc1eb1eb845ec82b79adee04a51e7)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:13.002418 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507e8e2-7118-4d3d-8132-9956d49be23b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:13.017373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:13.060624 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7507e8e2-7118-4d3d-8132-9956d49be23b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab2bc1eb1eb845ec82b79adee04a51e7'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:13.061227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7507e8e2-7118-4d3d-8132-9956d49be23b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab2bc1eb1eb845ec82b79adee04a51e7'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:13.073858 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7507e8e2-7118-4d3d-8132-9956d49be23b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab2bc1eb1eb845ec82b79adee04a51e7'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:13.081505 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7507e8e2-7118-4d3d-8132-9956d49be23b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab2bc1eb1eb845ec82b79adee04a51e7'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:13.095009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7507e8e2-7118-4d3d-8132-9956d49be23b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab2bc1eb1eb845ec82b79adee04a51e7'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:13.101252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1371/2698] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:05:12 2022] DELETE /identity/v3/users/ab2bc1eb1eb845ec82b79adee04a51e7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:13.282662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349e4db-3717-42bf-995a-efde18019053 service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:13.289157 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:13.300000 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349e4db-3717-42bf-995a-efde18019053 service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:13.359071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2d730-e7af-4879-bd0a-a74bea89fc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:13.359895 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2d730-e7af-4879-bd0a-a74bea89fc33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:13.360187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2d730-e7af-4879-bd0a-a74bea89fc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:13.360459 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2d730-e7af-4879-bd0a-a74bea89fc33 admin admin] PATH_INFO: `/v3/projects/c51d3542adae4062a487ab85d673140d` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:13.371734 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2d730-e7af-4879-bd0a-a74bea89fc33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c51d3542adae4062a487ab85d673140d)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:13.373700 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2d730-e7af-4879-bd0a-a74bea89fc33 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:13.423583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d9e2d730-e7af-4879-bd0a-a74bea89fc33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c51d3542adae4062a487ab85d673140d', 'request_id': 'req-d9e2d730-e7af-4879-bd0a-a74bea89fc33'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:13.432101 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d9e2d730-e7af-4879-bd0a-a74bea89fc33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c51d3542adae4062a487ab85d673140d', 'request_id': 'req-d9e2d730-e7af-4879-bd0a-a74bea89fc33'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:13.437770 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d9e2d730-e7af-4879-bd0a-a74bea89fc33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c51d3542adae4062a487ab85d673140d', 'request_id': 'req-d9e2d730-e7af-4879-bd0a-a74bea89fc33'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:13.450945 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d9e2d730-e7af-4879-bd0a-a74bea89fc33 admin admin] The token cache is being invalidate because project c51d3542adae4062a487ab85d673140d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:05:13.451293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d9e2d730-e7af-4879-bd0a-a74bea89fc33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:13.451978 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-d9e2d730-e7af-4879-bd0a-a74bea89fc33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:13.453667 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1328/2699] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:05:13 2022] DELETE /identity/v3/projects/c51d3542adae4062a487ab85d673140d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:13.458654 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cc75a-5add-483c-b60a-40af862ec128 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:13.458967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cc75a-5add-483c-b60a-40af862ec128 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:13.459249 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cc75a-5add-483c-b60a-40af862ec128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:13.478326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-1c6cc75a-5add-483c-b60a-40af862ec128 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:13.479881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1c6cc75a-5add-483c-b60a-40af862ec128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:13.541290 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1372/2700] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:13.549116 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8abdb-186c-4e3f-a189-9431a3c14840 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:13.549406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8abdb-186c-4e3f-a189-9431a3c14840 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:13.549683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8abdb-186c-4e3f-a189-9431a3c14840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:13.567192 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-b5b8abdb-186c-4e3f-a189-9431a3c14840 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:13.568431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b5b8abdb-186c-4e3f-a189-9431a3c14840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:13.576887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1329/2701] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:13.581051 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cc75a-5add-483c-b60a-40af862ec128 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:13.591611 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cc75a-5add-483c-b60a-40af862ec128 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:13.596233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9e30e-f8a2-4e4f-8655-789ecff8ca41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:13.596896 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9e30e-f8a2-4e4f-8655-789ecff8ca41 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:13.597184 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9e30e-f8a2-4e4f-8655-789ecff8ca41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:13.597464 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9e30e-f8a2-4e4f-8655-789ecff8ca41 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:13.598209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9e30e-f8a2-4e4f-8655-789ecff8ca41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:05:13.598517 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9e30e-f8a2-4e4f-8655-789ecff8ca41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:13.600142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9e30e-f8a2-4e4f-8655-789ecff8ca41 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:13.612339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1373/2702] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:05:13.616555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8abdb-186c-4e3f-a189-9431a3c14840 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:13.626768 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8abdb-186c-4e3f-a189-9431a3c14840 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:13.631307 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23b955-d9fd-4714-925f-67cec3506d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:13.632029 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23b955-d9fd-4714-925f-67cec3506d97 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:13.632311 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23b955-d9fd-4714-925f-67cec3506d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:13.632582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23b955-d9fd-4714-925f-67cec3506d97 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:13.633371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23b955-d9fd-4714-925f-67cec3506d97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:13.635239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23b955-d9fd-4714-925f-67cec3506d97 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:13.663750 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1330/2703] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:05:13 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:05:13.665587 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9e30e-f8a2-4e4f-8655-789ecff8ca41 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:13.675685 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9e30e-f8a2-4e4f-8655-789ecff8ca41 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:13.733971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd401654-255d-4155-ae2e-4b3b96f01df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:13.734639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd401654-255d-4155-ae2e-4b3b96f01df4 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:13.734927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd401654-255d-4155-ae2e-4b3b96f01df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:13.735208 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd401654-255d-4155-ae2e-4b3b96f01df4 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:13.735943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd401654-255d-4155-ae2e-4b3b96f01df4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:13.737844 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd401654-255d-4155-ae2e-4b3b96f01df4 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:13.757838 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1374/2704] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:05:13 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 21:05:13.762205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23b955-d9fd-4714-925f-67cec3506d97 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:13.772590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23b955-d9fd-4714-925f-67cec3506d97 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:13.778257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910af2f4-7457-4f0a-b925-e978887c9fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:13.778925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910af2f4-7457-4f0a-b925-e978887c9fdb admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:13.779318 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910af2f4-7457-4f0a-b925-e978887c9fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:13.779596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910af2f4-7457-4f0a-b925-e978887c9fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:13.780385 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910af2f4-7457-4f0a-b925-e978887c9fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:13.782045 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910af2f4-7457-4f0a-b925-e978887c9fdb admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:13.793124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1331/2705] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:13.797259 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd401654-255d-4155-ae2e-4b3b96f01df4 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:13.807471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd401654-255d-4155-ae2e-4b3b96f01df4 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:13.813245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a170e-6333-46bc-8d34-62ebfb890819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:13.813927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a170e-6333-46bc-8d34-62ebfb890819 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:13.814213 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a170e-6333-46bc-8d34-62ebfb890819 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:13.814492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a170e-6333-46bc-8d34-62ebfb890819 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:13.815221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a170e-6333-46bc-8d34-62ebfb890819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:13.816855 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a170e-6333-46bc-8d34-62ebfb890819 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:13.818907 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:13.829429 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1375/2706] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:13.833321 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910af2f4-7457-4f0a-b925-e978887c9fdb admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:13.844337 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910af2f4-7457-4f0a-b925-e978887c9fdb admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:13.848998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411d4e7-ef74-400f-9a71-6c4c33d12b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:13.849871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411d4e7-ef74-400f-9a71-6c4c33d12b80 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:13.850159 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411d4e7-ef74-400f-9a71-6c4c33d12b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:13.850438 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411d4e7-ef74-400f-9a71-6c4c33d12b80 admin admin] PATH_INFO: `/v3/projects/6549bea44b794d03bab57a0c2a001992/users/5272710cad204a49a52116becbce6bc1/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:13.865585 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411d4e7-ef74-400f-9a71-6c4c33d12b80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6549bea44b794d03bab57a0c2a001992, user_id=5272710cad204a49a52116becbce6bc1, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:13.868794 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411d4e7-ef74-400f-9a71-6c4c33d12b80 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:13.881597 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1332/2707] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:05:13 2022] PUT /identity/v3/projects/6549bea44b794d03bab57a0c2a001992/users/5272710cad204a49a52116becbce6bc1/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:13.887063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c0ff8-6a92-4af7-ba19-8abd03249add None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:13.887352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c0ff8-6a92-4af7-ba19-8abd03249add None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:13.887630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c0ff8-6a92-4af7-ba19-8abd03249add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:13.904956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-c28c0ff8-6a92-4af7-ba19-8abd03249add None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:13.906151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c28c0ff8-6a92-4af7-ba19-8abd03249add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:13.964705 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1376/2708] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:13.974282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411d4e7-ef74-400f-9a71-6c4c33d12b80 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:13.975987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-0411d4e7-ef74-400f-9a71-6c4c33d12b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:13.998292 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:14.034776 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411d4e7-ef74-400f-9a71-6c4c33d12b80 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:14.038510 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e77a7-1ce9-47c6-96f1-126b1db5264a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:14.039259 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e77a7-1ce9-47c6-96f1-126b1db5264a service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:14.039566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e77a7-1ce9-47c6-96f1-126b1db5264a service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:14.039878 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e77a7-1ce9-47c6-96f1-126b1db5264a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:14.048994 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e77a7-1ce9-47c6-96f1-126b1db5264a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:14.050424 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e77a7-1ce9-47c6-96f1-126b1db5264a service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:14.060564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1333/2709] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:05:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:05:14.683231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c0ff8-6a92-4af7-ba19-8abd03249add None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:14.684753 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-c28c0ff8-6a92-4af7-ba19-8abd03249add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:14.698899 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c0ff8-6a92-4af7-ba19-8abd03249add None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:14.704565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51076a0f-b0d4-42bd-8107-1ace5ad34075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:14.705459 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51076a0f-b0d4-42bd-8107-1ace5ad34075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:14.705745 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51076a0f-b0d4-42bd-8107-1ace5ad34075 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:14.706023 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51076a0f-b0d4-42bd-8107-1ace5ad34075 admin admin] PATH_INFO: `/v3/users/e515ec9a289542fea86506e43a3c5bc1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:14.707295 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51076a0f-b0d4-42bd-8107-1ace5ad34075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e515ec9a289542fea86506e43a3c5bc1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:14.709303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51076a0f-b0d4-42bd-8107-1ace5ad34075 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:14.793271 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-51076a0f-b0d4-42bd-8107-1ace5ad34075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e515ec9a289542fea86506e43a3c5bc1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:14.794315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-51076a0f-b0d4-42bd-8107-1ace5ad34075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e515ec9a289542fea86506e43a3c5bc1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:14.812578 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-51076a0f-b0d4-42bd-8107-1ace5ad34075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e515ec9a289542fea86506e43a3c5bc1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:14.820187 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-51076a0f-b0d4-42bd-8107-1ace5ad34075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e515ec9a289542fea86506e43a3c5bc1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:14.829148 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-51076a0f-b0d4-42bd-8107-1ace5ad34075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e515ec9a289542fea86506e43a3c5bc1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:14.839854 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1377/2710] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:05:14 2022] DELETE /identity/v3/users/e515ec9a289542fea86506e43a3c5bc1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:14.844481 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cc67d-bfe9-4f2b-90e5-b9bf6e6aabe8 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:14.845771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cc67d-bfe9-4f2b-90e5-b9bf6e6aabe8 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:14.846243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cc67d-bfe9-4f2b-90e5-b9bf6e6aabe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:14.868084 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c63cc67d-bfe9-4f2b-90e5-b9bf6e6aabe8 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:14.870072 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c63cc67d-bfe9-4f2b-90e5-b9bf6e6aabe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:14.960037 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1334/2711] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:14.971742 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51076a0f-b0d4-42bd-8107-1ace5ad34075 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:14.973230 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-51076a0f-b0d4-42bd-8107-1ace5ad34075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:15.045780 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51076a0f-b0d4-42bd-8107-1ace5ad34075 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:15.049926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf619f-5489-4836-bc39-2d828bdf32b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:15.051060 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf619f-5489-4836-bc39-2d828bdf32b6 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:15.051351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf619f-5489-4836-bc39-2d828bdf32b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:15.051635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf619f-5489-4836-bc39-2d828bdf32b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:15.063174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf619f-5489-4836-bc39-2d828bdf32b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:15.064692 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf619f-5489-4836-bc39-2d828bdf32b6 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:15.088877 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1378/2712] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:05:15.419102 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cc67d-bfe9-4f2b-90e5-b9bf6e6aabe8 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:15.420453 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c63cc67d-bfe9-4f2b-90e5-b9bf6e6aabe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:15.436410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cc67d-bfe9-4f2b-90e5-b9bf6e6aabe8 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:15.442051 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0761b-4bcb-4873-b82f-905ceed5a849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:15.442945 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0761b-4bcb-4873-b82f-905ceed5a849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:15.443486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0761b-4bcb-4873-b82f-905ceed5a849 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:15.443757 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0761b-4bcb-4873-b82f-905ceed5a849 admin admin] PATH_INFO: `/v3/projects/77c7cfe2e6954c3da403f754cfc9a52c` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:15.445685 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0761b-4bcb-4873-b82f-905ceed5a849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77c7cfe2e6954c3da403f754cfc9a52c)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:15.447777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0761b-4bcb-4873-b82f-905ceed5a849 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:15.514133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6dd0761b-4bcb-4873-b82f-905ceed5a849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77c7cfe2e6954c3da403f754cfc9a52c', 'request_id': 'req-6dd0761b-4bcb-4873-b82f-905ceed5a849'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:15.537042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6dd0761b-4bcb-4873-b82f-905ceed5a849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77c7cfe2e6954c3da403f754cfc9a52c', 'request_id': 'req-6dd0761b-4bcb-4873-b82f-905ceed5a849'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:15.545383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6dd0761b-4bcb-4873-b82f-905ceed5a849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77c7cfe2e6954c3da403f754cfc9a52c', 'request_id': 'req-6dd0761b-4bcb-4873-b82f-905ceed5a849'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:15.557671 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6dd0761b-4bcb-4873-b82f-905ceed5a849 admin admin] The token cache is being invalidate because project 77c7cfe2e6954c3da403f754cfc9a52c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:05:15.558068 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6dd0761b-4bcb-4873-b82f-905ceed5a849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:15.558810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6dd0761b-4bcb-4873-b82f-905ceed5a849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:15.560519 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1335/2713] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:05:15 2022] DELETE /identity/v3/projects/77c7cfe2e6954c3da403f754cfc9a52c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:15.564520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf619f-5489-4836-bc39-2d828bdf32b6 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:15.575635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf619f-5489-4836-bc39-2d828bdf32b6 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:15.586023 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:15.643565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2645f-e19a-49eb-a559-a1e610e59707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:15.644508 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2645f-e19a-49eb-a559-a1e610e59707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:15.644781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2645f-e19a-49eb-a559-a1e610e59707 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:15.645061 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2645f-e19a-49eb-a559-a1e610e59707 admin admin] PATH_INFO: `/v3/users/b44a09f74ea64024b6f766a4841c44bc` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:15.646316 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2645f-e19a-49eb-a559-a1e610e59707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b44a09f74ea64024b6f766a4841c44bc)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:15.648405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2645f-e19a-49eb-a559-a1e610e59707 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:15.715437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-5bf2645f-e19a-49eb-a559-a1e610e59707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b44a09f74ea64024b6f766a4841c44bc'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:15.716202 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-5bf2645f-e19a-49eb-a559-a1e610e59707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b44a09f74ea64024b6f766a4841c44bc'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:15.746832 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-5bf2645f-e19a-49eb-a559-a1e610e59707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b44a09f74ea64024b6f766a4841c44bc'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:15.754964 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-5bf2645f-e19a-49eb-a559-a1e610e59707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b44a09f74ea64024b6f766a4841c44bc'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:15.766340 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-5bf2645f-e19a-49eb-a559-a1e610e59707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b44a09f74ea64024b6f766a4841c44bc'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:15.778832 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1379/2714] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:05:15 2022] DELETE /identity/v3/users/b44a09f74ea64024b6f766a4841c44bc => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:05:16.078806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0761b-4bcb-4873-b82f-905ceed5a849 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:16.080496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6dd0761b-4bcb-4873-b82f-905ceed5a849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:16.095224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0761b-4bcb-4873-b82f-905ceed5a849 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:16.144602 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:16.160224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd64d9-7989-46f5-9b90-6241ee983622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:16.161269 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd64d9-7989-46f5-9b90-6241ee983622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:16.161559 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd64d9-7989-46f5-9b90-6241ee983622 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:16.161839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd64d9-7989-46f5-9b90-6241ee983622 admin admin] PATH_INFO: `/v3/projects/bd31eff7f1434830a1565c05eb1a42b8` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:16.163091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd64d9-7989-46f5-9b90-6241ee983622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd31eff7f1434830a1565c05eb1a42b8)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:16.165278 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd64d9-7989-46f5-9b90-6241ee983622 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:16.222407 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-abbd64d9-7989-46f5-9b90-6241ee983622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd31eff7f1434830a1565c05eb1a42b8', 'request_id': 'req-abbd64d9-7989-46f5-9b90-6241ee983622'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:16.231657 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-abbd64d9-7989-46f5-9b90-6241ee983622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd31eff7f1434830a1565c05eb1a42b8', 'request_id': 'req-abbd64d9-7989-46f5-9b90-6241ee983622'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:16.239613 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-abbd64d9-7989-46f5-9b90-6241ee983622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd31eff7f1434830a1565c05eb1a42b8', 'request_id': 'req-abbd64d9-7989-46f5-9b90-6241ee983622'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:16.261122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-abbd64d9-7989-46f5-9b90-6241ee983622 admin admin] The token cache is being invalidate because project bd31eff7f1434830a1565c05eb1a42b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:05:16.261342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-abbd64d9-7989-46f5-9b90-6241ee983622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:16.261996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-abbd64d9-7989-46f5-9b90-6241ee983622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:16.263901 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1336/2715] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:05:16 2022] DELETE /identity/v3/projects/bd31eff7f1434830a1565c05eb1a42b8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:16.267843 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2645f-e19a-49eb-a559-a1e610e59707 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:16.269329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-5bf2645f-e19a-49eb-a559-a1e610e59707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:16.290303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2645f-e19a-49eb-a559-a1e610e59707 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:16.361140 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d73be-1796-4e5b-bc21-81e4b027684e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:16.362028 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d73be-1796-4e5b-bc21-81e4b027684e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:16.362314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d73be-1796-4e5b-bc21-81e4b027684e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:16.362591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d73be-1796-4e5b-bc21-81e4b027684e admin admin] PATH_INFO: `/v3/users/e5a1a6768c1c4ac68d756c7dc2ba60ee` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:16.363867 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d73be-1796-4e5b-bc21-81e4b027684e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5a1a6768c1c4ac68d756c7dc2ba60ee)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:16.365808 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d73be-1796-4e5b-bc21-81e4b027684e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:16.423555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:16.449011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-4b2d73be-1796-4e5b-bc21-81e4b027684e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5a1a6768c1c4ac68d756c7dc2ba60ee'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:16.451275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-4b2d73be-1796-4e5b-bc21-81e4b027684e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5a1a6768c1c4ac68d756c7dc2ba60ee'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:16.474104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-4b2d73be-1796-4e5b-bc21-81e4b027684e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5a1a6768c1c4ac68d756c7dc2ba60ee'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:16.483949 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-4b2d73be-1796-4e5b-bc21-81e4b027684e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5a1a6768c1c4ac68d756c7dc2ba60ee'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:16.493073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-4b2d73be-1796-4e5b-bc21-81e4b027684e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5a1a6768c1c4ac68d756c7dc2ba60ee'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:16.500364 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1380/2716] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:05:16 2022] DELETE /identity/v3/users/e5a1a6768c1c4ac68d756c7dc2ba60ee => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:05:16.923200 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd64d9-7989-46f5-9b90-6241ee983622 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:16.924932 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-abbd64d9-7989-46f5-9b90-6241ee983622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:16.938084 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd64d9-7989-46f5-9b90-6241ee983622 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:17.047072 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8133f-156c-46c5-b2a8-0875caa28e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:17.048254 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8133f-156c-46c5-b2a8-0875caa28e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:17.048638 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8133f-156c-46c5-b2a8-0875caa28e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:17.049009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8133f-156c-46c5-b2a8-0875caa28e5c admin admin] PATH_INFO: `/v3/projects/700a5011aecf4d5b80a0176b1f4c9a01` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:17.050479 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8133f-156c-46c5-b2a8-0875caa28e5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=700a5011aecf4d5b80a0176b1f4c9a01)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:17.052608 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8133f-156c-46c5-b2a8-0875caa28e5c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:17.132922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-87b8133f-156c-46c5-b2a8-0875caa28e5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '700a5011aecf4d5b80a0176b1f4c9a01', 'request_id': 'req-87b8133f-156c-46c5-b2a8-0875caa28e5c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:17.143874 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-87b8133f-156c-46c5-b2a8-0875caa28e5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '700a5011aecf4d5b80a0176b1f4c9a01', 'request_id': 'req-87b8133f-156c-46c5-b2a8-0875caa28e5c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:17.152905 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-87b8133f-156c-46c5-b2a8-0875caa28e5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '700a5011aecf4d5b80a0176b1f4c9a01', 'request_id': 'req-87b8133f-156c-46c5-b2a8-0875caa28e5c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:17.167207 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-87b8133f-156c-46c5-b2a8-0875caa28e5c admin admin] The token cache is being invalidate because project 700a5011aecf4d5b80a0176b1f4c9a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:05:17.167552 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-87b8133f-156c-46c5-b2a8-0875caa28e5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:17.168276 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-87b8133f-156c-46c5-b2a8-0875caa28e5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:17.169881 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1337/2717] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:05:16 2022] DELETE /identity/v3/projects/700a5011aecf4d5b80a0176b1f4c9a01 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:17.175978 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440cae0-9141-4de9-b0f3-c0f1c2f418e2 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:17.176275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440cae0-9141-4de9-b0f3-c0f1c2f418e2 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:17.176565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440cae0-9141-4de9-b0f3-c0f1c2f418e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:17.197953 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-7440cae0-9141-4de9-b0f3-c0f1c2f418e2 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:17.199255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-7440cae0-9141-4de9-b0f3-c0f1c2f418e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:17.275330 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1381/2718] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:17.285816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4490b2-541c-403d-95a8-110e9709d3e3 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:17.287110 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4490b2-541c-403d-95a8-110e9709d3e3 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:17.287447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4490b2-541c-403d-95a8-110e9709d3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:17.305710 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:17.315412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-db4490b2-541c-403d-95a8-110e9709d3e3 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:17.317909 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-db4490b2-541c-403d-95a8-110e9709d3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:17.330253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1338/2719] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:17.331542 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440cae0-9141-4de9-b0f3-c0f1c2f418e2 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:17.344130 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440cae0-9141-4de9-b0f3-c0f1c2f418e2 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:17.350445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be3fdd-5c0d-4d3e-9756-c53302d86629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:17.351895 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be3fdd-5c0d-4d3e-9756-c53302d86629 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:17.352406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be3fdd-5c0d-4d3e-9756-c53302d86629 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:17.352791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be3fdd-5c0d-4d3e-9756-c53302d86629 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:17.353644 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be3fdd-5c0d-4d3e-9756-c53302d86629 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:05:17.354968 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be3fdd-5c0d-4d3e-9756-c53302d86629 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:17.355906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be3fdd-5c0d-4d3e-9756-c53302d86629 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:17.374524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1382/2720] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:05:17.376161 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4490b2-541c-403d-95a8-110e9709d3e3 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:17.386791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4490b2-541c-403d-95a8-110e9709d3e3 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:17.394152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ccd55-ee4d-4af1-80b2-a3120e03003d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:17.394965 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ccd55-ee4d-4af1-80b2-a3120e03003d admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:17.395661 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ccd55-ee4d-4af1-80b2-a3120e03003d admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:17.396344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ccd55-ee4d-4af1-80b2-a3120e03003d admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:17.397156 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ccd55-ee4d-4af1-80b2-a3120e03003d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:17.399762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ccd55-ee4d-4af1-80b2-a3120e03003d admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:17.434990 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1339/2721] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:05:17 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:05:17.438375 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be3fdd-5c0d-4d3e-9756-c53302d86629 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:17.452396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be3fdd-5c0d-4d3e-9756-c53302d86629 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:17.538543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363b454-99f1-4e2d-a169-b92b5042644f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:17.539750 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363b454-99f1-4e2d-a169-b92b5042644f admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:17.540072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363b454-99f1-4e2d-a169-b92b5042644f admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:17.540347 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363b454-99f1-4e2d-a169-b92b5042644f admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:17.541077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363b454-99f1-4e2d-a169-b92b5042644f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:17.543125 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363b454-99f1-4e2d-a169-b92b5042644f admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:17.570337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1383/2722] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:05:17 2022] POST /identity/v3/users => generated 295 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:05:17.571026 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ccd55-ee4d-4af1-80b2-a3120e03003d admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:17.583631 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ccd55-ee4d-4af1-80b2-a3120e03003d admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:17.589640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb4932-0b1a-413a-97fe-182bfb4fc2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:17.590808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb4932-0b1a-413a-97fe-182bfb4fc2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:17.591180 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb4932-0b1a-413a-97fe-182bfb4fc2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:17.591551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb4932-0b1a-413a-97fe-182bfb4fc2c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:17.592542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb4932-0b1a-413a-97fe-182bfb4fc2c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:17.594336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb4932-0b1a-413a-97fe-182bfb4fc2c1 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:17.611791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1340/2723] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:17.613204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363b454-99f1-4e2d-a169-b92b5042644f admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:17.624928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363b454-99f1-4e2d-a169-b92b5042644f admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:17.631239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e498d-54af-4875-9e5d-d5056c43d45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:17.631975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e498d-54af-4875-9e5d-d5056c43d45b admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:17.632242 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e498d-54af-4875-9e5d-d5056c43d45b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:17.632518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e498d-54af-4875-9e5d-d5056c43d45b admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:17.635304 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e498d-54af-4875-9e5d-d5056c43d45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:17.635304 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e498d-54af-4875-9e5d-d5056c43d45b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:17.658576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1384/2724] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:17.658788 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb4932-0b1a-413a-97fe-182bfb4fc2c1 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:17.670723 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb4932-0b1a-413a-97fe-182bfb4fc2c1 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:17.676004 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd3a78-eddd-4193-bf90-f5c54725fc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:17.676910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd3a78-eddd-4193-bf90-f5c54725fc2c admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:17.677197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd3a78-eddd-4193-bf90-f5c54725fc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:17.677479 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd3a78-eddd-4193-bf90-f5c54725fc2c admin admin] PATH_INFO: `/v3/projects/027d691ffed64abca15b26f2994c6a29/users/abb53cae0f9e461da1e5bb645d9381b0/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:17.696743 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd3a78-eddd-4193-bf90-f5c54725fc2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=027d691ffed64abca15b26f2994c6a29, user_id=abb53cae0f9e461da1e5bb645d9381b0, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:17.704005 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd3a78-eddd-4193-bf90-f5c54725fc2c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:17.722098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1341/2725] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:05:17 2022] PUT /identity/v3/projects/027d691ffed64abca15b26f2994c6a29/users/abb53cae0f9e461da1e5bb645d9381b0/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:17.733790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367e758-ff88-4e7d-8e51-3daf89b4d4e4 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:17.734131 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367e758-ff88-4e7d-8e51-3daf89b4d4e4 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:17.734392 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367e758-ff88-4e7d-8e51-3daf89b4d4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:17.779378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-9367e758-ff88-4e7d-8e51-3daf89b4d4e4 None None] MFA Rules not processed for user `abb53cae0f9e461da1e5bb645d9381b0`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:17.780864 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9367e758-ff88-4e7d-8e51-3daf89b4d4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:17.790173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:17.873373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1385/2726] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:17.877917 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd3a78-eddd-4193-bf90-f5c54725fc2c admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:17.961721 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd3a78-eddd-4193-bf90-f5c54725fc2c admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:17.966817 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf2f11-b566-49bf-b07d-7168a8255d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:17.967494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf2f11-b566-49bf-b07d-7168a8255d68 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:17.967807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf2f11-b566-49bf-b07d-7168a8255d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:17.968244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf2f11-b566-49bf-b07d-7168a8255d68 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:17.968952 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf2f11-b566-49bf-b07d-7168a8255d68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:17.970906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf2f11-b566-49bf-b07d-7168a8255d68 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:18.005324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1342/2727] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:05:17 2022] POST /identity/v3/projects => generated 389 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 21:05:18.006510 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367e758-ff88-4e7d-8e51-3daf89b4d4e4 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:18.018971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367e758-ff88-4e7d-8e51-3daf89b4d4e4 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:18.095602 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0045e8-2754-4239-9d2d-7e8d8c459d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:18.096346 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0045e8-2754-4239-9d2d-7e8d8c459d94 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:18.096630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0045e8-2754-4239-9d2d-7e8d8c459d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:18.096905 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0045e8-2754-4239-9d2d-7e8d8c459d94 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:18.097840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0045e8-2754-4239-9d2d-7e8d8c459d94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:18.099504 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0045e8-2754-4239-9d2d-7e8d8c459d94 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:18.127832 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1386/2728] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:05:18 2022] POST /identity/v3/users => generated 301 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:05:18.129449 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf2f11-b566-49bf-b07d-7168a8255d68 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:18.142248 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf2f11-b566-49bf-b07d-7168a8255d68 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:18.149702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56098b-4fe5-498a-b9aa-b28038deee94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:18.150668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56098b-4fe5-498a-b9aa-b28038deee94 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:18.150795 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56098b-4fe5-498a-b9aa-b28038deee94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:18.151065 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56098b-4fe5-498a-b9aa-b28038deee94 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:18.151832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56098b-4fe5-498a-b9aa-b28038deee94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:18.153719 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56098b-4fe5-498a-b9aa-b28038deee94 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:18.169862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1343/2729] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:18.171053 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0045e8-2754-4239-9d2d-7e8d8c459d94 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:18.182413 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0045e8-2754-4239-9d2d-7e8d8c459d94 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:18.188471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0dec5-ab39-40ab-a38d-46a68c8e0a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:18.190006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0dec5-ab39-40ab-a38d-46a68c8e0a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:18.190342 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0dec5-ab39-40ab-a38d-46a68c8e0a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:18.190624 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0dec5-ab39-40ab-a38d-46a68c8e0a04 admin admin] PATH_INFO: `/v3/projects/2d75e2b1b67c410384afa46fdb29cefd/users/d7659fdad88a4a469c2bcb98945c62e4/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:18.213230 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0dec5-ab39-40ab-a38d-46a68c8e0a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d75e2b1b67c410384afa46fdb29cefd, user_id=d7659fdad88a4a469c2bcb98945c62e4, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:18.216147 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0dec5-ab39-40ab-a38d-46a68c8e0a04 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:18.237185 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1387/2730] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:05:18 2022] PUT /identity/v3/projects/2d75e2b1b67c410384afa46fdb29cefd/users/d7659fdad88a4a469c2bcb98945c62e4/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:18.240885 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56098b-4fe5-498a-b9aa-b28038deee94 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:18.252197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56098b-4fe5-498a-b9aa-b28038deee94 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:18.336693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e90230-501d-47f2-8e7a-dfaacf6dfddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:18.337891 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e90230-501d-47f2-8e7a-dfaacf6dfddd admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:18.338369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e90230-501d-47f2-8e7a-dfaacf6dfddd admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:18.338810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e90230-501d-47f2-8e7a-dfaacf6dfddd admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:18.339744 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e90230-501d-47f2-8e7a-dfaacf6dfddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:18.341946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e90230-501d-47f2-8e7a-dfaacf6dfddd admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:18.360193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1344/2731] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 21:05:18.364014 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0dec5-ab39-40ab-a38d-46a68c8e0a04 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:18.375541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0dec5-ab39-40ab-a38d-46a68c8e0a04 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:18.380657 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735bfed8-85e0-4e13-8a9e-8c66750172cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:18.381664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bfed8-85e0-4e13-8a9e-8c66750172cf admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:18.382061 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bfed8-85e0-4e13-8a9e-8c66750172cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:18.382548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bfed8-85e0-4e13-8a9e-8c66750172cf admin admin] PATH_INFO: `/v3/projects/2d75e2b1b67c410384afa46fdb29cefd/users/d7659fdad88a4a469c2bcb98945c62e4/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:18.385842 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735bfed8-85e0-4e13-8a9e-8c66750172cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d75e2b1b67c410384afa46fdb29cefd, user_id=d7659fdad88a4a469c2bcb98945c62e4, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:18.389123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735bfed8-85e0-4e13-8a9e-8c66750172cf admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:18.401954 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1388/2732] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:05:18 2022] PUT /identity/v3/projects/2d75e2b1b67c410384afa46fdb29cefd/users/d7659fdad88a4a469c2bcb98945c62e4/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:18.412176 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020c3b7-a298-45b4-b4be-ccbace268ca2 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:18.412632 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020c3b7-a298-45b4-b4be-ccbace268ca2 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:18.418620 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020c3b7-a298-45b4-b4be-ccbace268ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:18.452146 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:18.457592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-1020c3b7-a298-45b4-b4be-ccbace268ca2 None None] MFA Rules not processed for user `d7659fdad88a4a469c2bcb98945c62e4`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:18.459452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1020c3b7-a298-45b4-b4be-ccbace268ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:18.503811 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba822a-6160-4b6e-966d-fd154eab4cf4 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:18.504123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba822a-6160-4b6e-966d-fd154eab4cf4 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:18.504406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba822a-6160-4b6e-966d-fd154eab4cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:18.548199 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e2ba822a-6160-4b6e-966d-fd154eab4cf4 None None] MFA Rules not processed for user `5272710cad204a49a52116becbce6bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:18.549567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e2ba822a-6160-4b6e-966d-fd154eab4cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:18.558240 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1345/2733] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:18.567407 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020c3b7-a298-45b4-b4be-ccbace268ca2 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:18.568817 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1020c3b7-a298-45b4-b4be-ccbace268ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:18.613275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:18.651499 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1389/2734] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 21:05:18.656382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020c3b7-a298-45b4-b4be-ccbace268ca2 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:18.660546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fb7d8-b85d-4ebe-b204-2f4f3be32f1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:18.661202 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fb7d8-b85d-4ebe-b204-2f4f3be32f1c service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:18.662187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fb7d8-b85d-4ebe-b204-2f4f3be32f1c service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:18.662489 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fb7d8-b85d-4ebe-b204-2f4f3be32f1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:18.666511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba822a-6160-4b6e-966d-fd154eab4cf4 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:18.667923 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e2ba822a-6160-4b6e-966d-fd154eab4cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:18.674719 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764fb7d8-b85d-4ebe-b204-2f4f3be32f1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:18.676292 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764fb7d8-b85d-4ebe-b204-2f4f3be32f1c service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:18.689359 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1346/2735] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:05:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 21:05:18.776957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba822a-6160-4b6e-966d-fd154eab4cf4 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:18.783005 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f045dd-a62b-4122-931a-7d80af7d486d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:18.783791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f045dd-a62b-4122-931a-7d80af7d486d service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:18.784272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f045dd-a62b-4122-931a-7d80af7d486d service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:18.784625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f045dd-a62b-4122-931a-7d80af7d486d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:18.794980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f045dd-a62b-4122-931a-7d80af7d486d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:18.797036 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f045dd-a62b-4122-931a-7d80af7d486d service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:18.811197 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1390/2736] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:05:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 21:05:18.844076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fb7d8-b85d-4ebe-b204-2f4f3be32f1c service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:18.938701 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fb7d8-b85d-4ebe-b204-2f4f3be32f1c service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:18.946735 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0512613-c5e3-46f7-ad38-82d43d7148e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:18.947893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0512613-c5e3-46f7-ad38-82d43d7148e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:18.948282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0512613-c5e3-46f7-ad38-82d43d7148e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:18.949074 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0512613-c5e3-46f7-ad38-82d43d7148e1 admin admin] PATH_INFO: `/v3/users/abb53cae0f9e461da1e5bb645d9381b0` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:18.950271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0512613-c5e3-46f7-ad38-82d43d7148e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abb53cae0f9e461da1e5bb645d9381b0)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:18.952393 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0512613-c5e3-46f7-ad38-82d43d7148e1 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:19.008839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b0512613-c5e3-46f7-ad38-82d43d7148e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abb53cae0f9e461da1e5bb645d9381b0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:19.011908 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:19.030394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b0512613-c5e3-46f7-ad38-82d43d7148e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abb53cae0f9e461da1e5bb645d9381b0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:19.031102 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b0512613-c5e3-46f7-ad38-82d43d7148e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abb53cae0f9e461da1e5bb645d9381b0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:19.040461 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b0512613-c5e3-46f7-ad38-82d43d7148e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abb53cae0f9e461da1e5bb645d9381b0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:19.041462 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b0512613-c5e3-46f7-ad38-82d43d7148e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abb53cae0f9e461da1e5bb645d9381b0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:19.066434 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1347/2737] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:05:18 2022] DELETE /identity/v3/users/abb53cae0f9e461da1e5bb645d9381b0 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:05:19.071612 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a653b-0712-4759-bc2c-6173de8d8156 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:19.072063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a653b-0712-4759-bc2c-6173de8d8156 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:19.072354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a653b-0712-4759-bc2c-6173de8d8156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:19.095183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-281a653b-0712-4759-bc2c-6173de8d8156 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:19.096840 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-281a653b-0712-4759-bc2c-6173de8d8156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:19.179955 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1391/2738] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:19.188722 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0512613-c5e3-46f7-ad38-82d43d7148e1 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:19.190461 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b0512613-c5e3-46f7-ad38-82d43d7148e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:19.269961 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0512613-c5e3-46f7-ad38-82d43d7148e1 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:19.274570 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91cf785-270b-415f-b392-9c505d71dc26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:19.275366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cf785-270b-415f-b392-9c505d71dc26 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:19.275655 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cf785-270b-415f-b392-9c505d71dc26 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:19.275966 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cf785-270b-415f-b392-9c505d71dc26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:19.285913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91cf785-270b-415f-b392-9c505d71dc26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:19.287380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91cf785-270b-415f-b392-9c505d71dc26 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:19.301462 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1348/2739] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:05:19.656804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a653b-0712-4759-bc2c-6173de8d8156 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:19.658543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-281a653b-0712-4759-bc2c-6173de8d8156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:19.673439 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a653b-0712-4759-bc2c-6173de8d8156 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:19.678548 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c287ab-10f3-4ae0-a1cf-a943517f0481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:19.680326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c287ab-10f3-4ae0-a1cf-a943517f0481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:19.680737 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c287ab-10f3-4ae0-a1cf-a943517f0481 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:19.681123 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c287ab-10f3-4ae0-a1cf-a943517f0481 admin admin] PATH_INFO: `/v3/projects/027d691ffed64abca15b26f2994c6a29` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:19.684291 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c287ab-10f3-4ae0-a1cf-a943517f0481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=027d691ffed64abca15b26f2994c6a29)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:19.686888 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c287ab-10f3-4ae0-a1cf-a943517f0481 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:19.748304 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45c287ab-10f3-4ae0-a1cf-a943517f0481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '027d691ffed64abca15b26f2994c6a29', 'request_id': 'req-45c287ab-10f3-4ae0-a1cf-a943517f0481'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:19.754972 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45c287ab-10f3-4ae0-a1cf-a943517f0481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '027d691ffed64abca15b26f2994c6a29', 'request_id': 'req-45c287ab-10f3-4ae0-a1cf-a943517f0481'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:19.762928 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45c287ab-10f3-4ae0-a1cf-a943517f0481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '027d691ffed64abca15b26f2994c6a29', 'request_id': 'req-45c287ab-10f3-4ae0-a1cf-a943517f0481'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:19.772783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:19.780084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45c287ab-10f3-4ae0-a1cf-a943517f0481 admin admin] The token cache is being invalidate because project 027d691ffed64abca15b26f2994c6a29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:05:19.780619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45c287ab-10f3-4ae0-a1cf-a943517f0481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:19.781957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-45c287ab-10f3-4ae0-a1cf-a943517f0481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:19.785563 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1392/2740] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:05:19 2022] DELETE /identity/v3/projects/027d691ffed64abca15b26f2994c6a29 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:19.791481 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91cf785-270b-415f-b392-9c505d71dc26 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:19.808212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91cf785-270b-415f-b392-9c505d71dc26 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:19.881019 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360118da-788a-4a7b-933a-4052913f73c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:19.881579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360118da-788a-4a7b-933a-4052913f73c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:19.881973 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360118da-788a-4a7b-933a-4052913f73c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:19.882358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360118da-788a-4a7b-933a-4052913f73c8 admin admin] PATH_INFO: `/v3/users/d7659fdad88a4a469c2bcb98945c62e4` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:19.884871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360118da-788a-4a7b-933a-4052913f73c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7659fdad88a4a469c2bcb98945c62e4)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:19.886831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360118da-788a-4a7b-933a-4052913f73c8 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:19.950592 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-360118da-788a-4a7b-933a-4052913f73c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7659fdad88a4a469c2bcb98945c62e4'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:19.958459 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-360118da-788a-4a7b-933a-4052913f73c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7659fdad88a4a469c2bcb98945c62e4'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:19.959136 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-360118da-788a-4a7b-933a-4052913f73c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7659fdad88a4a469c2bcb98945c62e4'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:19.971217 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-360118da-788a-4a7b-933a-4052913f73c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7659fdad88a4a469c2bcb98945c62e4'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:19.972380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-360118da-788a-4a7b-933a-4052913f73c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7659fdad88a4a469c2bcb98945c62e4'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:19.974131 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:19.995388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1349/2741] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:05:19 2022] DELETE /identity/v3/users/d7659fdad88a4a469c2bcb98945c62e4 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:20.438705 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e848a6b-20f1-4d83-b85e-313f8f52d17d None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:20.438999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e848a6b-20f1-4d83-b85e-313f8f52d17d None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:20.439279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e848a6b-20f1-4d83-b85e-313f8f52d17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:20.457229 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-4e848a6b-20f1-4d83-b85e-313f8f52d17d None None] MFA Rules not processed for user `5272710cad204a49a52116becbce6bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:20.458453 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-4e848a6b-20f1-4d83-b85e-313f8f52d17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:20.487494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360118da-788a-4a7b-933a-4052913f73c8 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:20.489441 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-360118da-788a-4a7b-933a-4052913f73c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:20.542165 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1393/2742] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:20.568168 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360118da-788a-4a7b-933a-4052913f73c8 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:20.573576 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0609a7-a5c2-4d33-9329-f6f0131ba867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:20.574443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0609a7-a5c2-4d33-9329-f6f0131ba867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:20.575234 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0609a7-a5c2-4d33-9329-f6f0131ba867 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:20.575234 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0609a7-a5c2-4d33-9329-f6f0131ba867 admin admin] PATH_INFO: `/v3/projects/2d75e2b1b67c410384afa46fdb29cefd` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:20.576304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0609a7-a5c2-4d33-9329-f6f0131ba867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d75e2b1b67c410384afa46fdb29cefd)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:20.581824 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0609a7-a5c2-4d33-9329-f6f0131ba867 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:20.655211 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-3b0609a7-a5c2-4d33-9329-f6f0131ba867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d75e2b1b67c410384afa46fdb29cefd', 'request_id': 'req-3b0609a7-a5c2-4d33-9329-f6f0131ba867'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:20.667615 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-3b0609a7-a5c2-4d33-9329-f6f0131ba867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d75e2b1b67c410384afa46fdb29cefd', 'request_id': 'req-3b0609a7-a5c2-4d33-9329-f6f0131ba867'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:20.684329 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-3b0609a7-a5c2-4d33-9329-f6f0131ba867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d75e2b1b67c410384afa46fdb29cefd', 'request_id': 'req-3b0609a7-a5c2-4d33-9329-f6f0131ba867'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:20.698923 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-3b0609a7-a5c2-4d33-9329-f6f0131ba867 admin admin] The token cache is being invalidate because project 2d75e2b1b67c410384afa46fdb29cefd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:05:20.699522 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-3b0609a7-a5c2-4d33-9329-f6f0131ba867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:20.700353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-3b0609a7-a5c2-4d33-9329-f6f0131ba867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:20.702985 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1350/2743] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:05:20 2022] DELETE /identity/v3/projects/2d75e2b1b67c410384afa46fdb29cefd => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:20.708642 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38207333-ad0d-40b5-8b80-31c66cbb9dca None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:20.709105 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38207333-ad0d-40b5-8b80-31c66cbb9dca None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:20.709334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38207333-ad0d-40b5-8b80-31c66cbb9dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:20.731665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-38207333-ad0d-40b5-8b80-31c66cbb9dca None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:20.733105 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-38207333-ad0d-40b5-8b80-31c66cbb9dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:20.814363 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1394/2744] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:20.822113 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf12db2-cd51-4988-9c32-63760a7b6414 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:20.822402 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf12db2-cd51-4988-9c32-63760a7b6414 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:20.822681 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf12db2-cd51-4988-9c32-63760a7b6414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:20.848441 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-eaf12db2-cd51-4988-9c32-63760a7b6414 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:20.849783 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-eaf12db2-cd51-4988-9c32-63760a7b6414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:20.863030 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1351/2745] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:20.867000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38207333-ad0d-40b5-8b80-31c66cbb9dca None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:20.882118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38207333-ad0d-40b5-8b80-31c66cbb9dca None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:20.888175 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1ff57-4c1b-4952-a2d3-ac16a4d04971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:20.888868 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1ff57-4c1b-4952-a2d3-ac16a4d04971 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:20.889220 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1ff57-4c1b-4952-a2d3-ac16a4d04971 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:20.889562 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1ff57-4c1b-4952-a2d3-ac16a4d04971 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:20.890567 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1ff57-4c1b-4952-a2d3-ac16a4d04971 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:05:20.891000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1ff57-4c1b-4952-a2d3-ac16a4d04971 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:20.892761 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1ff57-4c1b-4952-a2d3-ac16a4d04971 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:20.910616 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1395/2746] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:05:20.911517 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf12db2-cd51-4988-9c32-63760a7b6414 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:20.925733 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf12db2-cd51-4988-9c32-63760a7b6414 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:20.935297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c5fdb-430c-41ef-a5e7-ea74262ef163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:20.936396 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c5fdb-430c-41ef-a5e7-ea74262ef163 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:20.936953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c5fdb-430c-41ef-a5e7-ea74262ef163 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:20.938077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c5fdb-430c-41ef-a5e7-ea74262ef163 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:20.939834 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c5fdb-430c-41ef-a5e7-ea74262ef163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:20.942356 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c5fdb-430c-41ef-a5e7-ea74262ef163 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:20.974324 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1352/2747] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:05:20 2022] POST /identity/v3/projects => generated 399 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 21:05:20.979225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1ff57-4c1b-4952-a2d3-ac16a4d04971 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:20.989207 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1ff57-4c1b-4952-a2d3-ac16a4d04971 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:21.058342 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56cd25-7841-4228-aabd-f3a8f5c438c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:21.059020 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56cd25-7841-4228-aabd-f3a8f5c438c8 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:21.059301 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56cd25-7841-4228-aabd-f3a8f5c438c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:21.059577 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56cd25-7841-4228-aabd-f3a8f5c438c8 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:21.060315 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56cd25-7841-4228-aabd-f3a8f5c438c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:21.062255 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56cd25-7841-4228-aabd-f3a8f5c438c8 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:21.087070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1396/2748] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:05:20 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:05:21.089581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c5fdb-430c-41ef-a5e7-ea74262ef163 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:21.102530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c5fdb-430c-41ef-a5e7-ea74262ef163 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:21.108241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092b825-3ffa-4a8a-9a10-c725c22e867d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:21.109002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092b825-3ffa-4a8a-9a10-c725c22e867d admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:21.109353 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092b825-3ffa-4a8a-9a10-c725c22e867d admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:21.109746 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092b825-3ffa-4a8a-9a10-c725c22e867d admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:21.110550 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092b825-3ffa-4a8a-9a10-c725c22e867d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:21.112397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092b825-3ffa-4a8a-9a10-c725c22e867d admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:21.129024 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1353/2749] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:21.133594 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56cd25-7841-4228-aabd-f3a8f5c438c8 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:21.143962 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56cd25-7841-4228-aabd-f3a8f5c438c8 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:21.149647 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b78e8-9942-48e7-94a5-c368ee109a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:21.150679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b78e8-9942-48e7-94a5-c368ee109a3b admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:21.150679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b78e8-9942-48e7-94a5-c368ee109a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:21.150972 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b78e8-9942-48e7-94a5-c368ee109a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:21.151639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b78e8-9942-48e7-94a5-c368ee109a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:21.153637 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b78e8-9942-48e7-94a5-c368ee109a3b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:21.168307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1397/2750] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:21.169081 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092b825-3ffa-4a8a-9a10-c725c22e867d admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:21.181383 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092b825-3ffa-4a8a-9a10-c725c22e867d admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:21.186791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b202e-2700-4774-9246-5ce00c4ae1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:21.187862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b202e-2700-4774-9246-5ce00c4ae1c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:21.188241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b202e-2700-4774-9246-5ce00c4ae1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:21.189419 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b202e-2700-4774-9246-5ce00c4ae1c1 admin admin] PATH_INFO: `/v3/projects/c573da9c5b414e338e9acda904b6f5fc/users/92e60f00204d48fc83c62ebcc7e72a68/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:21.209564 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b202e-2700-4774-9246-5ce00c4ae1c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c573da9c5b414e338e9acda904b6f5fc, user_id=92e60f00204d48fc83c62ebcc7e72a68, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:21.213633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b202e-2700-4774-9246-5ce00c4ae1c1 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:21.238554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1354/2751] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:05:21 2022] PUT /identity/v3/projects/c573da9c5b414e338e9acda904b6f5fc/users/92e60f00204d48fc83c62ebcc7e72a68/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:05:21.250384 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f69b1-eb23-43a8-86b3-8ca06f5cc7b0 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:21.250689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f69b1-eb23-43a8-86b3-8ca06f5cc7b0 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:21.250945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f69b1-eb23-43a8-86b3-8ca06f5cc7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:21.275227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:21.287744 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-601f69b1-eb23-43a8-86b3-8ca06f5cc7b0 None None] MFA Rules not processed for user `92e60f00204d48fc83c62ebcc7e72a68`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:21.289868 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-601f69b1-eb23-43a8-86b3-8ca06f5cc7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:21.373841 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1398/2752] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:21.387173 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b202e-2700-4774-9246-5ce00c4ae1c1 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:21.389090 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-4e6b202e-2700-4774-9246-5ce00c4ae1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:21.402096 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:21.471345 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b202e-2700-4774-9246-5ce00c4ae1c1 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:21.475735 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f2ec8-4fff-4711-b670-dd927be529da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:21.477179 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f2ec8-4fff-4711-b670-dd927be529da service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:21.477555 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f2ec8-4fff-4711-b670-dd927be529da service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:21.477922 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f2ec8-4fff-4711-b670-dd927be529da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:21.487139 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f2ec8-4fff-4711-b670-dd927be529da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:21.488607 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f2ec8-4fff-4711-b670-dd927be529da service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:21.500270 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1355/2753] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:05:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:05:31.370629 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=28 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:31.495065 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=22 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:31.714843 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f69b1-eb23-43a8-86b3-8ca06f5cc7b0 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:31.792152 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f69b1-eb23-43a8-86b3-8ca06f5cc7b0 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:31.799539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07826e-a0c0-4e3a-ae5e-f8135677c22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:31.800633 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07826e-a0c0-4e3a-ae5e-f8135677c22b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:31.801482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07826e-a0c0-4e3a-ae5e-f8135677c22b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:31.802240 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07826e-a0c0-4e3a-ae5e-f8135677c22b admin admin] PATH_INFO: `/v3/users/92e60f00204d48fc83c62ebcc7e72a68` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:31.803451 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07826e-a0c0-4e3a-ae5e-f8135677c22b admin admin] RBAC: Authorizing `identity:delete_user(user_id=92e60f00204d48fc83c62ebcc7e72a68)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:31.805556 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07826e-a0c0-4e3a-ae5e-f8135677c22b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:31.865933 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-4d07826e-a0c0-4e3a-ae5e-f8135677c22b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92e60f00204d48fc83c62ebcc7e72a68'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:31.866697 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-4d07826e-a0c0-4e3a-ae5e-f8135677c22b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92e60f00204d48fc83c62ebcc7e72a68'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:31.881948 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-4d07826e-a0c0-4e3a-ae5e-f8135677c22b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92e60f00204d48fc83c62ebcc7e72a68'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:31.894215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-4d07826e-a0c0-4e3a-ae5e-f8135677c22b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92e60f00204d48fc83c62ebcc7e72a68'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:31.904229 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-4d07826e-a0c0-4e3a-ae5e-f8135677c22b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92e60f00204d48fc83c62ebcc7e72a68'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:31.914091 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1399/2754] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:05:31 2022] DELETE /identity/v3/users/92e60f00204d48fc83c62ebcc7e72a68 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:31.919341 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fe9c2-209b-4514-b6d7-33d0f2ab63c1 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:31.919640 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fe9c2-209b-4514-b6d7-33d0f2ab63c1 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:31.919953 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fe9c2-209b-4514-b6d7-33d0f2ab63c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:31.946184 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-1c3fe9c2-209b-4514-b6d7-33d0f2ab63c1 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:31.948267 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1c3fe9c2-209b-4514-b6d7-33d0f2ab63c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:32.015998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1356/2755] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:32.026852 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07826e-a0c0-4e3a-ae5e-f8135677c22b admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:32.028480 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-4d07826e-a0c0-4e3a-ae5e-f8135677c22b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:32.096411 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07826e-a0c0-4e3a-ae5e-f8135677c22b admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:32.100682 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4e585-2a48-4b10-accb-4feae02eecc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:32.101350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4e585-2a48-4b10-accb-4feae02eecc3 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:32.101645 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4e585-2a48-4b10-accb-4feae02eecc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:32.101913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4e585-2a48-4b10-accb-4feae02eecc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:32.112048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4e585-2a48-4b10-accb-4feae02eecc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:32.113512 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4e585-2a48-4b10-accb-4feae02eecc3 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:32.125312 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1400/2756] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 21:05:32.417451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fe9c2-209b-4514-b6d7-33d0f2ab63c1 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:32.419355 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1c3fe9c2-209b-4514-b6d7-33d0f2ab63c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:32.440400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fe9c2-209b-4514-b6d7-33d0f2ab63c1 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:32.445823 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5fcf5-85e2-459f-b63f-735e47374dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:32.447303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5fcf5-85e2-459f-b63f-735e47374dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:32.447577 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5fcf5-85e2-459f-b63f-735e47374dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:32.447962 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5fcf5-85e2-459f-b63f-735e47374dd9 admin admin] PATH_INFO: `/v3/projects/c573da9c5b414e338e9acda904b6f5fc` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:32.449530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5fcf5-85e2-459f-b63f-735e47374dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c573da9c5b414e338e9acda904b6f5fc)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:32.452669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5fcf5-85e2-459f-b63f-735e47374dd9 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:32.503473 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-33b5fcf5-85e2-459f-b63f-735e47374dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c573da9c5b414e338e9acda904b6f5fc', 'request_id': 'req-33b5fcf5-85e2-459f-b63f-735e47374dd9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:32.512389 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-33b5fcf5-85e2-459f-b63f-735e47374dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c573da9c5b414e338e9acda904b6f5fc', 'request_id': 'req-33b5fcf5-85e2-459f-b63f-735e47374dd9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:32.519392 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-33b5fcf5-85e2-459f-b63f-735e47374dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c573da9c5b414e338e9acda904b6f5fc', 'request_id': 'req-33b5fcf5-85e2-459f-b63f-735e47374dd9'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:32.531808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-33b5fcf5-85e2-459f-b63f-735e47374dd9 admin admin] The token cache is being invalidate because project c573da9c5b414e338e9acda904b6f5fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:05:32.532105 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-33b5fcf5-85e2-459f-b63f-735e47374dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:32.532799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-33b5fcf5-85e2-459f-b63f-735e47374dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:05:32.534510 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1357/2757] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:05:32 2022] DELETE /identity/v3/projects/c573da9c5b414e338e9acda904b6f5fc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:05:32.543221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6601e7-317f-4177-8536-e8a7e73ad089 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:32.543801 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6601e7-317f-4177-8536-e8a7e73ad089 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:32.543917 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6601e7-317f-4177-8536-e8a7e73ad089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:32.562955 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-ee6601e7-317f-4177-8536-e8a7e73ad089 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:32.564229 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ee6601e7-317f-4177-8536-e8a7e73ad089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:32.582226 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:32.646456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1401/2758] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:32.655277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3291766-3d1a-4b37-b9ed-4839b1a8aa7b None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:32.655512 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3291766-3d1a-4b37-b9ed-4839b1a8aa7b None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:32.655810 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3291766-3d1a-4b37-b9ed-4839b1a8aa7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:32.675262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-d3291766-3d1a-4b37-b9ed-4839b1a8aa7b None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:32.676519 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-d3291766-3d1a-4b37-b9ed-4839b1a8aa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:32.685516 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1358/2759] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:32.689978 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6601e7-317f-4177-8536-e8a7e73ad089 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:32.700954 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6601e7-317f-4177-8536-e8a7e73ad089 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:32.706208 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38189030-afb3-4888-9fb4-011646ff4062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:32.706873 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38189030-afb3-4888-9fb4-011646ff4062 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:32.707154 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38189030-afb3-4888-9fb4-011646ff4062 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:32.707434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38189030-afb3-4888-9fb4-011646ff4062 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:32.708193 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38189030-afb3-4888-9fb4-011646ff4062 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:05:32.708526 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38189030-afb3-4888-9fb4-011646ff4062 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:32.710120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38189030-afb3-4888-9fb4-011646ff4062 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:32.726048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1402/2760] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:05:32.727151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3291766-3d1a-4b37-b9ed-4839b1a8aa7b None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:32.736472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3291766-3d1a-4b37-b9ed-4839b1a8aa7b None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:32.741128 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddf171-5880-4cd9-84ca-618bcc07ef88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:32.741783 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddf171-5880-4cd9-84ca-618bcc07ef88 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:32.742063 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddf171-5880-4cd9-84ca-618bcc07ef88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:32.742340 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddf171-5880-4cd9-84ca-618bcc07ef88 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:32.743285 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cddf171-5880-4cd9-84ca-618bcc07ef88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:32.745481 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cddf171-5880-4cd9-84ca-618bcc07ef88 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:32.774534 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1359/2761] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:05:32 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:05:32.781839 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38189030-afb3-4888-9fb4-011646ff4062 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:32.792509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38189030-afb3-4888-9fb4-011646ff4062 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:32.858327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f141a2-c3f6-495e-b36c-339b5dbba2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:32.859011 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f141a2-c3f6-495e-b36c-339b5dbba2ed admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:32.859296 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f141a2-c3f6-495e-b36c-339b5dbba2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:32.859575 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f141a2-c3f6-495e-b36c-339b5dbba2ed admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:32.860365 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f141a2-c3f6-495e-b36c-339b5dbba2ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:32.862337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f141a2-c3f6-495e-b36c-339b5dbba2ed admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:32.888995 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1403/2762] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:05:32 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:05:32.890706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddf171-5880-4cd9-84ca-618bcc07ef88 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:32.902036 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddf171-5880-4cd9-84ca-618bcc07ef88 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:32.907024 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d297af6-fb90-4deb-8c55-81eb4e1f3885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:32.907723 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d297af6-fb90-4deb-8c55-81eb4e1f3885 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:32.908016 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d297af6-fb90-4deb-8c55-81eb4e1f3885 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:32.908289 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d297af6-fb90-4deb-8c55-81eb4e1f3885 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:32.909027 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d297af6-fb90-4deb-8c55-81eb4e1f3885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:32.910620 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d297af6-fb90-4deb-8c55-81eb4e1f3885 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:32.923480 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1360/2763] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:32.927401 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f141a2-c3f6-495e-b36c-339b5dbba2ed admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:32.938079 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f141a2-c3f6-495e-b36c-339b5dbba2ed admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:32.942669 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695710a-a7d5-4d77-9c6a-dcc20ceb4456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:32.943354 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695710a-a7d5-4d77-9c6a-dcc20ceb4456 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:32.943636 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695710a-a7d5-4d77-9c6a-dcc20ceb4456 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:32.943942 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695710a-a7d5-4d77-9c6a-dcc20ceb4456 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:32.944691 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695710a-a7d5-4d77-9c6a-dcc20ceb4456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:32.946390 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695710a-a7d5-4d77-9c6a-dcc20ceb4456 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:32.959756 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1404/2764] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:32.961927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d297af6-fb90-4deb-8c55-81eb4e1f3885 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:32.972955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d297af6-fb90-4deb-8c55-81eb4e1f3885 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:32.977831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ba1ac-7879-4fa5-8089-26f02ff521e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:32.979621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ba1ac-7879-4fa5-8089-26f02ff521e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:32.980025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ba1ac-7879-4fa5-8089-26f02ff521e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:32.980233 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ba1ac-7879-4fa5-8089-26f02ff521e7 admin admin] PATH_INFO: `/v3/projects/91eaf4caa6ea457bafa7de7a1387851c/users/f06aae5f002d46a3a60100c908d07455/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:32.998911 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ba1ac-7879-4fa5-8089-26f02ff521e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91eaf4caa6ea457bafa7de7a1387851c, user_id=f06aae5f002d46a3a60100c908d07455, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:33.001761 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ba1ac-7879-4fa5-8089-26f02ff521e7 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:33.021057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1361/2765] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:05:32 2022] PUT /identity/v3/projects/91eaf4caa6ea457bafa7de7a1387851c/users/f06aae5f002d46a3a60100c908d07455/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:33.026779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331f6a1-1f37-4ad9-a6e3-0f6ca6b08a92 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:33.027042 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331f6a1-1f37-4ad9-a6e3-0f6ca6b08a92 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:33.027329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331f6a1-1f37-4ad9-a6e3-0f6ca6b08a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:33.046539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-1331f6a1-1f37-4ad9-a6e3-0f6ca6b08a92 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:33.047853 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1331f6a1-1f37-4ad9-a6e3-0f6ca6b08a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:33.119402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1405/2766] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:33.130260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ba1ac-7879-4fa5-8089-26f02ff521e7 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:33.136195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-db1ba1ac-7879-4fa5-8089-26f02ff521e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:33.170354 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:33.212535 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ba1ac-7879-4fa5-8089-26f02ff521e7 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:33.216371 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cade842-8ce8-4836-9dc9-efbcb0754868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:33.217039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cade842-8ce8-4836-9dc9-efbcb0754868 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:33.217313 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cade842-8ce8-4836-9dc9-efbcb0754868 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:33.217594 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cade842-8ce8-4836-9dc9-efbcb0754868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:33.226829 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cade842-8ce8-4836-9dc9-efbcb0754868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:33.228832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cade842-8ce8-4836-9dc9-efbcb0754868 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:33.238418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1362/2767] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:05:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:05:37.446059 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a5d67-7183-408c-bfeb-c929e2400ef7 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:37.446588 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a5d67-7183-408c-bfeb-c929e2400ef7 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:37.446776 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a5d67-7183-408c-bfeb-c929e2400ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:37.474913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-6a4a5d67-7183-408c-bfeb-c929e2400ef7 None None] MFA Rules not processed for user `f06aae5f002d46a3a60100c908d07455`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:37.476062 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6a4a5d67-7183-408c-bfeb-c929e2400ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:37.496538 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:37.546523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1406/2768] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:37.551143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cade842-8ce8-4836-9dc9-efbcb0754868 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:37.561776 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cade842-8ce8-4836-9dc9-efbcb0754868 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:37.567022 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1c3b8-3b04-410d-8156-7f74a396c699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:37.567806 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1c3b8-3b04-410d-8156-7f74a396c699 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:37.568195 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1c3b8-3b04-410d-8156-7f74a396c699 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:37.568562 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1c3b8-3b04-410d-8156-7f74a396c699 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:37.569342 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1c3b8-3b04-410d-8156-7f74a396c699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:37.571310 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1c3b8-3b04-410d-8156-7f74a396c699 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:37.602010 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1363/2769] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:05:37 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:05:37.603243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a5d67-7183-408c-bfeb-c929e2400ef7 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:37.614070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a5d67-7183-408c-bfeb-c929e2400ef7 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:37.675784 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e6c30-adb0-4baa-9cc0-adcd06853467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:37.676474 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e6c30-adb0-4baa-9cc0-adcd06853467 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:37.677209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e6c30-adb0-4baa-9cc0-adcd06853467 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:37.677209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e6c30-adb0-4baa-9cc0-adcd06853467 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:37.677722 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e6c30-adb0-4baa-9cc0-adcd06853467 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:37.679832 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e6c30-adb0-4baa-9cc0-adcd06853467 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:37.704798 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1407/2770] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:05:37 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 21:05:37.705971 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1c3b8-3b04-410d-8156-7f74a396c699 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:37.717707 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1c3b8-3b04-410d-8156-7f74a396c699 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:37.722565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e859e8-ca25-445c-887d-d3fb0c085e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:37.723380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e859e8-ca25-445c-887d-d3fb0c085e2f admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:37.723666 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e859e8-ca25-445c-887d-d3fb0c085e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:37.723967 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e859e8-ca25-445c-887d-d3fb0c085e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:37.724851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e859e8-ca25-445c-887d-d3fb0c085e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:37.726494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e859e8-ca25-445c-887d-d3fb0c085e2f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:37.742047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1364/2771] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:37.746795 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e6c30-adb0-4baa-9cc0-adcd06853467 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:37.757431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e6c30-adb0-4baa-9cc0-adcd06853467 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:37.763283 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a6ec7-f170-4327-9ce7-f000837204bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:37.764319 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a6ec7-f170-4327-9ce7-f000837204bd admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:37.764779 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a6ec7-f170-4327-9ce7-f000837204bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:37.765072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a6ec7-f170-4327-9ce7-f000837204bd admin admin] PATH_INFO: `/v3/projects/4bd5a82e19f0453fa0585af78fc35e6a/users/cb292a64c14d4e66b974e9f9d1022d66/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:37.782775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a6ec7-f170-4327-9ce7-f000837204bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd5a82e19f0453fa0585af78fc35e6a, user_id=cb292a64c14d4e66b974e9f9d1022d66, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:37.785533 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a6ec7-f170-4327-9ce7-f000837204bd admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:37.801564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1408/2772] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:05:37 2022] PUT /identity/v3/projects/4bd5a82e19f0453fa0585af78fc35e6a/users/cb292a64c14d4e66b974e9f9d1022d66/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:37.805510 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e859e8-ca25-445c-887d-d3fb0c085e2f admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:37.816668 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e859e8-ca25-445c-887d-d3fb0c085e2f admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:37.876499 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bee3b-98fd-4d10-911d-87fd23202e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:37.877171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bee3b-98fd-4d10-911d-87fd23202e7c admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:37.877452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bee3b-98fd-4d10-911d-87fd23202e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:37.877807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bee3b-98fd-4d10-911d-87fd23202e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:37.878434 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bee3b-98fd-4d10-911d-87fd23202e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:37.880061 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bee3b-98fd-4d10-911d-87fd23202e7c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:37.891880 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1365/2773] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:05:37.902340 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a6ec7-f170-4327-9ce7-f000837204bd admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:37.919554 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a6ec7-f170-4327-9ce7-f000837204bd admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:37.924491 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ce8c5-3444-48f7-a5ab-5125a3f8288a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:37.925470 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ce8c5-3444-48f7-a5ab-5125a3f8288a admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:37.925755 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ce8c5-3444-48f7-a5ab-5125a3f8288a admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:37.926095 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ce8c5-3444-48f7-a5ab-5125a3f8288a admin admin] PATH_INFO: `/v3/projects/4bd5a82e19f0453fa0585af78fc35e6a/users/cb292a64c14d4e66b974e9f9d1022d66/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:37.928159 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ce8c5-3444-48f7-a5ab-5125a3f8288a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd5a82e19f0453fa0585af78fc35e6a, user_id=cb292a64c14d4e66b974e9f9d1022d66, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:37.930977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ce8c5-3444-48f7-a5ab-5125a3f8288a admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:37.945884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1409/2774] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:05:37 2022] PUT /identity/v3/projects/4bd5a82e19f0453fa0585af78fc35e6a/users/cb292a64c14d4e66b974e9f9d1022d66/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:05:43.533485 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776f503-6c75-4773-9ee6-c449939bd2da None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:43.534045 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776f503-6c75-4773-9ee6-c449939bd2da None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:43.534045 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776f503-6c75-4773-9ee6-c449939bd2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:43.560603 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-2776f503-6c75-4773-9ee6-c449939bd2da None None] MFA Rules not processed for user `cb292a64c14d4e66b974e9f9d1022d66`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:43.561748 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-2776f503-6c75-4773-9ee6-c449939bd2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:43.634839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1366/2775] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:43.649217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ce8c5-3444-48f7-a5ab-5125a3f8288a admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:05:43.651733 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e63ce8c5-3444-48f7-a5ab-5125a3f8288a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:43.736993 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ce8c5-3444-48f7-a5ab-5125a3f8288a admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:05:43.742794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e520f8-90b6-47a0-b4e8-d205635411ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:05:43.744174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e520f8-90b6-47a0-b4e8-d205635411ab service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:43.744461 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e520f8-90b6-47a0-b4e8-d205635411ab service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:43.744738 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e520f8-90b6-47a0-b4e8-d205635411ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:43.750478 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:43.802119 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e520f8-90b6-47a0-b4e8-d205635411ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:05:43.803738 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e520f8-90b6-47a0-b4e8-d205635411ab service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:05:43.816373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1410/2776] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:05:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:05:44.957140 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d352bd0-2aa9-4897-93c5-8f7d4e05e1b0 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:05:44.957669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d352bd0-2aa9-4897-93c5-8f7d4e05e1b0 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:05:44.957669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d352bd0-2aa9-4897-93c5-8f7d4e05e1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:05:44.976836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-1d352bd0-2aa9-4897-93c5-8f7d4e05e1b0 None None] MFA Rules not processed for user `f06aae5f002d46a3a60100c908d07455`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:05:44.978688 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1d352bd0-2aa9-4897-93c5-8f7d4e05e1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:05:44.986228 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1367/2777] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:05:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:05:53.810227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=24 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:05:54.971218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:06:11.008151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e520f8-90b6-47a0-b4e8-d205635411ab service nova] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:06:11.009576 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e1e520f8-90b6-47a0-b4e8-d205635411ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:06:11.066739 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e520f8-90b6-47a0-b4e8-d205635411ab service nova] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:06:11.070330 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee8242-0af8-4025-871d-13eb52fcacc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:06:11.070965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee8242-0af8-4025-871d-13eb52fcacc9 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:06:11.071245 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee8242-0af8-4025-871d-13eb52fcacc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:06:11.071520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee8242-0af8-4025-871d-13eb52fcacc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:06:11.073368 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-53ee8242-0af8-4025-871d-13eb52fcacc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:06:11.082783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee8242-0af8-4025-871d-13eb52fcacc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:06:11.084243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee8242-0af8-4025-871d-13eb52fcacc9 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:06:11.092777 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1411/2778] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:06:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:06:12.029194 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d352bd0-2aa9-4897-93c5-8f7d4e05e1b0 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:06:12.030686 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-1d352bd0-2aa9-4897-93c5-8f7d4e05e1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:06:12.102720 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d352bd0-2aa9-4897-93c5-8f7d4e05e1b0 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:06:12.108454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7df7da-a794-4255-a6e0-b32ba2d66c8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:06:12.109112 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7df7da-a794-4255-a6e0-b32ba2d66c8e service placement] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:06:12.109400 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7df7da-a794-4255-a6e0-b32ba2d66c8e service placement] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:06:12.109679 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7df7da-a794-4255-a6e0-b32ba2d66c8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:06:12.111470 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6f7df7da-a794-4255-a6e0-b32ba2d66c8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:06:12.124441 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7df7da-a794-4255-a6e0-b32ba2d66c8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:06:12.126087 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7df7da-a794-4255-a6e0-b32ba2d66c8e service placement] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:06:12.136857 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1368/2779] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:06:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:06:21.089334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=26 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:06:21.111939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee8242-0af8-4025-871d-13eb52fcacc9 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:06:21.121585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee8242-0af8-4025-871d-13eb52fcacc9 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:06:21.125299 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396cc1ed-a256-4353-8248-fe8fa80209b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:06:21.125940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396cc1ed-a256-4353-8248-fe8fa80209b1 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:06:21.126227 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396cc1ed-a256-4353-8248-fe8fa80209b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:06:21.126504 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396cc1ed-a256-4353-8248-fe8fa80209b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:06:21.128289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-396cc1ed-a256-4353-8248-fe8fa80209b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:06:21.207978 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396cc1ed-a256-4353-8248-fe8fa80209b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:06:21.209412 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396cc1ed-a256-4353-8248-fe8fa80209b1 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:06:21.227969 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1412/2780] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:06:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:06:22.131949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=32 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:06:29.507802 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7df7da-a794-4255-a6e0-b32ba2d66c8e service placement] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:06:29.518295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7df7da-a794-4255-a6e0-b32ba2d66c8e service placement] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:06:29.522060 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d0d5f-c38f-4027-b530-ed4315956c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:06:29.522705 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d0d5f-c38f-4027-b530-ed4315956c05 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:06:29.523027 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d0d5f-c38f-4027-b530-ed4315956c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:06:29.523257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d0d5f-c38f-4027-b530-ed4315956c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:06:29.533576 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d0d5f-c38f-4027-b530-ed4315956c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:06:29.535006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d0d5f-c38f-4027-b530-ed4315956c05 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:06:29.545034 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1369/2781] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:06:29 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:06:31.225353 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=41 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:06:31.314094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396cc1ed-a256-4353-8248-fe8fa80209b1 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:06:31.316240 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-396cc1ed-a256-4353-8248-fe8fa80209b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:06:31.333836 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396cc1ed-a256-4353-8248-fe8fa80209b1 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:06:31.340990 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073ef32-d388-4ac5-8fc2-d0af6252d00d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:06:31.341805 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073ef32-d388-4ac5-8fc2-d0af6252d00d service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:06:31.342191 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073ef32-d388-4ac5-8fc2-d0af6252d00d service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:06:31.343010 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073ef32-d388-4ac5-8fc2-d0af6252d00d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:06:31.345001 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-6073ef32-d388-4ac5-8fc2-d0af6252d00d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:06:31.360120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073ef32-d388-4ac5-8fc2-d0af6252d00d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:06:31.361909 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073ef32-d388-4ac5-8fc2-d0af6252d00d service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:06:31.389679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1413/2782] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:06:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:06:35.202443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d0d5f-c38f-4027-b530-ed4315956c05 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:06:35.211440 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d0d5f-c38f-4027-b530-ed4315956c05 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:06:35.215859 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8be6d-7332-4ec4-a42b-b45429992da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:06:35.216706 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8be6d-7332-4ec4-a42b-b45429992da7 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:06:35.216984 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8be6d-7332-4ec4-a42b-b45429992da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:06:35.217266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8be6d-7332-4ec4-a42b-b45429992da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:06:35.219061 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5cc8be6d-7332-4ec4-a42b-b45429992da7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:06:35.283302 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8be6d-7332-4ec4-a42b-b45429992da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:06:35.284777 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8be6d-7332-4ec4-a42b-b45429992da7 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:06:35.295170 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1370/2783] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:06:38.135287 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073ef32-d388-4ac5-8fc2-d0af6252d00d service nova] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:06:38.147950 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073ef32-d388-4ac5-8fc2-d0af6252d00d service nova] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:06:38.154478 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dca94-4161-49e0-89a0-462331b2b61e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:06:38.155480 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dca94-4161-49e0-89a0-462331b2b61e service placement] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:06:38.155695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dca94-4161-49e0-89a0-462331b2b61e service placement] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:06:38.155978 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dca94-4161-49e0-89a0-462331b2b61e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:06:38.158471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-695dca94-4161-49e0-89a0-462331b2b61e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:06:38.182689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695dca94-4161-49e0-89a0-462331b2b61e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:06:38.186662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695dca94-4161-49e0-89a0-462331b2b61e service placement] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:06:38.206388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1414/2784] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:06:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 21:06:41.067492 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8be6d-7332-4ec4-a42b-b45429992da7 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:06:41.080636 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8be6d-7332-4ec4-a42b-b45429992da7 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:06:41.086265 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09facb-8ef6-40ac-8146-757eca2c42b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:06:41.087243 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09facb-8ef6-40ac-8146-757eca2c42b1 service placement] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:06:41.087804 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09facb-8ef6-40ac-8146-757eca2c42b1 service placement] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:06:41.088332 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09facb-8ef6-40ac-8146-757eca2c42b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:06:41.091246 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5d09facb-8ef6-40ac-8146-757eca2c42b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:06:41.109486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09facb-8ef6-40ac-8146-757eca2c42b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:06:41.111399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09facb-8ef6-40ac-8146-757eca2c42b1 service placement] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:06:41.126161 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1371/2785] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:06:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:06:41.146682 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dca94-4161-49e0-89a0-462331b2b61e service placement] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:06:41.148158 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-695dca94-4161-49e0-89a0-462331b2b61e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:06:41.168957 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dca94-4161-49e0-89a0-462331b2b61e service placement] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:06:41.174593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f12382-158d-4b58-b1e8-3ebf91599f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:06:41.175531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f12382-158d-4b58-b1e8-3ebf91599f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:06:41.175809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f12382-158d-4b58-b1e8-3ebf91599f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:06:41.176084 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f12382-158d-4b58-b1e8-3ebf91599f7d admin admin] PATH_INFO: `/v3/users/2b3617b8bcac4227b1295e294eec664a` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:06:41.177293 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f12382-158d-4b58-b1e8-3ebf91599f7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b3617b8bcac4227b1295e294eec664a)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:06:41.179239 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f12382-158d-4b58-b1e8-3ebf91599f7d admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:06:41.228569 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-95f12382-158d-4b58-b1e8-3ebf91599f7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b3617b8bcac4227b1295e294eec664a'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:41.229221 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-95f12382-158d-4b58-b1e8-3ebf91599f7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b3617b8bcac4227b1295e294eec664a'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:41.241737 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-95f12382-158d-4b58-b1e8-3ebf91599f7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b3617b8bcac4227b1295e294eec664a'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:41.249599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-95f12382-158d-4b58-b1e8-3ebf91599f7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b3617b8bcac4227b1295e294eec664a'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:41.256558 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-95f12382-158d-4b58-b1e8-3ebf91599f7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b3617b8bcac4227b1295e294eec664a'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:41.262618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1415/2786] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:06:41 2022] DELETE /identity/v3/users/2b3617b8bcac4227b1295e294eec664a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:06:41.444186 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09facb-8ef6-40ac-8146-757eca2c42b1 service placement] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:06:41.456064 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09facb-8ef6-40ac-8146-757eca2c42b1 service placement] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:06:41.514536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:06:41.515397 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:06:41.515678 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:06:41.515988 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0 admin admin] PATH_INFO: `/v3/projects/f2d1bc3b35054c44bbe955a9eddc3052` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:06:41.517197 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2d1bc3b35054c44bbe955a9eddc3052)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:06:41.519107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:06:41.565494 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2d1bc3b35054c44bbe955a9eddc3052', 'request_id': 'req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:41.573375 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2d1bc3b35054c44bbe955a9eddc3052', 'request_id': 'req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:41.578915 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2d1bc3b35054c44bbe955a9eddc3052', 'request_id': 'req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:41.585812 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:06:41.590380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0 admin admin] The token cache is being invalidate because project f2d1bc3b35054c44bbe955a9eddc3052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:06:41.590731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:41.591941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:41.593558 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1372/2787] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:06:41 2022] DELETE /identity/v3/projects/f2d1bc3b35054c44bbe955a9eddc3052 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:06:41.597594 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f12382-158d-4b58-b1e8-3ebf91599f7d admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:06:41.599002 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-95f12382-158d-4b58-b1e8-3ebf91599f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:06:41.613194 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f12382-158d-4b58-b1e8-3ebf91599f7d admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:06:41.672524 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9569671-ddb1-4130-b793-31f343b2cc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:06:41.673393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9569671-ddb1-4130-b793-31f343b2cc61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:06:41.673675 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9569671-ddb1-4130-b793-31f343b2cc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:06:41.673951 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9569671-ddb1-4130-b793-31f343b2cc61 admin admin] PATH_INFO: `/v3/users/fe45d7e58a064f69ba18ed69b47c2050` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:06:41.811675 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9569671-ddb1-4130-b793-31f343b2cc61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe45d7e58a064f69ba18ed69b47c2050)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:06:41.813635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9569671-ddb1-4130-b793-31f343b2cc61 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:06:41.851367 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:06:41.861788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b9569671-ddb1-4130-b793-31f343b2cc61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe45d7e58a064f69ba18ed69b47c2050'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:41.862768 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b9569671-ddb1-4130-b793-31f343b2cc61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe45d7e58a064f69ba18ed69b47c2050'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:41.875790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b9569671-ddb1-4130-b793-31f343b2cc61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe45d7e58a064f69ba18ed69b47c2050'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:41.882781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b9569671-ddb1-4130-b793-31f343b2cc61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe45d7e58a064f69ba18ed69b47c2050'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:41.890595 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b9569671-ddb1-4130-b793-31f343b2cc61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe45d7e58a064f69ba18ed69b47c2050'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:41.896656 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1416/2788] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:06:41 2022] DELETE /identity/v3/users/fe45d7e58a064f69ba18ed69b47c2050 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:06:42.072085 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:06:42.073587 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:06:42.094165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae02c09-dc58-43fe-987c-d1a2e9953fb0 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:06:42.151808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:06:42.153293 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:06:42.153575 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:06:42.153853 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3 admin admin] PATH_INFO: `/v3/projects/fec83747abfe4021a8b4336d17d0c214` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:06:42.164319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fec83747abfe4021a8b4336d17d0c214)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:06:42.166281 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:06:42.213465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fec83747abfe4021a8b4336d17d0c214', 'request_id': 'req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:42.221301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fec83747abfe4021a8b4336d17d0c214', 'request_id': 'req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:42.227580 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fec83747abfe4021a8b4336d17d0c214', 'request_id': 'req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:42.239596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3 admin admin] The token cache is being invalidate because project fec83747abfe4021a8b4336d17d0c214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:06:42.239913 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:42.240669 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:06:42.242229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1373/2789] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:06:42 2022] DELETE /identity/v3/projects/fec83747abfe4021a8b4336d17d0c214 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:06:51.893057 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=15,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:06:52.236629 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=56,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:07:04.212333 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b07922-3ec0-40bc-8a4e-49eb11206a03 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:07:04.212846 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b07922-3ec0-40bc-8a4e-49eb11206a03 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:07:04.212960 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b07922-3ec0-40bc-8a4e-49eb11206a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:07:04.233702 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-e8b07922-3ec0-40bc-8a4e-49eb11206a03 None None] MFA Rules not processed for user `f0beb65e5dc7469eb17a98dae1f09a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:07:04.234899 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e8b07922-3ec0-40bc-8a4e-49eb11206a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:07:04.288180 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1417/2790] 10.222.0.60 () {58 vars in 967 bytes} [Thu Jul 21 21:07:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:07:04.328290 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:07:04.329722 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:07:04.398730 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c1e9a-7c18-4854-b1a0-09c9f4f07ba3 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:07:04.404244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cf4f8-2ec2-468f-92cc-451372649c5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '833a474f828f4349bea0d1649461c77e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:07:04.404897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cf4f8-2ec2-468f-92cc-451372649c5e service swift] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:07:04.405229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cf4f8-2ec2-468f-92cc-451372649c5e service swift] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:07:04.405508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cf4f8-2ec2-468f-92cc-451372649c5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:07:04.413799 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3cf4f8-2ec2-468f-92cc-451372649c5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:07:04.415380 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3cf4f8-2ec2-468f-92cc-451372649c5e service swift] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:07:04.423556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1374/2791] 10.222.0.60 () {60 vars in 1384 bytes} [Thu Jul 21 21:07:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 21:07:10.820791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b07922-3ec0-40bc-8a4e-49eb11206a03 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:07:10.822246 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-e8b07922-3ec0-40bc-8a4e-49eb11206a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:07:10.901219 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b07922-3ec0-40bc-8a4e-49eb11206a03 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:07:10.907250 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd1e33-baf1-440c-a691-7c69819336a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:07:10.908326 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd1e33-baf1-440c-a691-7c69819336a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:07:10.908598 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd1e33-baf1-440c-a691-7c69819336a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:07:10.908880 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd1e33-baf1-440c-a691-7c69819336a4 admin admin] PATH_INFO: `/v3/users/084923b3deae41cd89c3935aed11a566` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:07:10.910445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd1e33-baf1-440c-a691-7c69819336a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=084923b3deae41cd89c3935aed11a566)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:07:10.912463 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd1e33-baf1-440c-a691-7c69819336a4 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:07:10.961809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-78cd1e33-baf1-440c-a691-7c69819336a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '084923b3deae41cd89c3935aed11a566'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:10.962456 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-78cd1e33-baf1-440c-a691-7c69819336a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '084923b3deae41cd89c3935aed11a566'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:10.974771 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-78cd1e33-baf1-440c-a691-7c69819336a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '084923b3deae41cd89c3935aed11a566'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:10.981922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-78cd1e33-baf1-440c-a691-7c69819336a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '084923b3deae41cd89c3935aed11a566'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:10.988589 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-78cd1e33-baf1-440c-a691-7c69819336a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '084923b3deae41cd89c3935aed11a566'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:10.994578 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1418/2792] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:07:10 2022] DELETE /identity/v3/users/084923b3deae41cd89c3935aed11a566 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:07:11.172026 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cf4f8-2ec2-468f-92cc-451372649c5e service swift] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:07:11.182635 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cf4f8-2ec2-468f-92cc-451372649c5e service swift] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:07:11.243078 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:07:11.244244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:07:11.244544 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:07:11.244814 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd admin admin] PATH_INFO: `/v3/projects/6a26de2015bb4626ba4093aac7a3d085` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:07:11.246633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a26de2015bb4626ba4093aac7a3d085)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:07:11.248581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:07:11.294897 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a26de2015bb4626ba4093aac7a3d085', 'request_id': 'req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:11.302388 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a26de2015bb4626ba4093aac7a3d085', 'request_id': 'req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:11.308369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a26de2015bb4626ba4093aac7a3d085', 'request_id': 'req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:11.319281 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd admin admin] The token cache is being invalidate because project 6a26de2015bb4626ba4093aac7a3d085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:07:11.319562 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:11.320212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:11.321728 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1375/2793] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:07:11 2022] DELETE /identity/v3/projects/6a26de2015bb4626ba4093aac7a3d085 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:07:11.326504 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd1e33-baf1-440c-a691-7c69819336a4 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:07:11.327948 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-78cd1e33-baf1-440c-a691-7c69819336a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:07:11.341001 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd1e33-baf1-440c-a691-7c69819336a4 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:07:11.360777 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:07:11.398789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f309a-1ff8-4272-a878-0f5598a2489b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:07:11.399672 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f309a-1ff8-4272-a878-0f5598a2489b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:07:11.399987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f309a-1ff8-4272-a878-0f5598a2489b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:07:11.400265 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f309a-1ff8-4272-a878-0f5598a2489b admin admin] PATH_INFO: `/v3/users/cfe2160787d644fd85fca7ba4d7bfff4` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:07:11.401489 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f309a-1ff8-4272-a878-0f5598a2489b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe2160787d644fd85fca7ba4d7bfff4)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:07:11.403552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f309a-1ff8-4272-a878-0f5598a2489b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:07:11.448560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7a5f309a-1ff8-4272-a878-0f5598a2489b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe2160787d644fd85fca7ba4d7bfff4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:11.449322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7a5f309a-1ff8-4272-a878-0f5598a2489b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe2160787d644fd85fca7ba4d7bfff4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:11.461001 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7a5f309a-1ff8-4272-a878-0f5598a2489b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe2160787d644fd85fca7ba4d7bfff4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:11.468459 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7a5f309a-1ff8-4272-a878-0f5598a2489b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe2160787d644fd85fca7ba4d7bfff4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:11.475235 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7a5f309a-1ff8-4272-a878-0f5598a2489b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe2160787d644fd85fca7ba4d7bfff4'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:11.481745 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1419/2794] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:07:11 2022] DELETE /identity/v3/users/cfe2160787d644fd85fca7ba4d7bfff4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:07:11.652805 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:07:11.654271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:07:11.668178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f5c5f-7a1f-4058-b55a-d6d028f32afd admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:07:11.687104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:07:11.729205 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431017b6-f8d4-43f1-aa72-ad54957fd942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:07:11.730658 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431017b6-f8d4-43f1-aa72-ad54957fd942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:07:11.730939 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431017b6-f8d4-43f1-aa72-ad54957fd942 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:07:11.731216 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431017b6-f8d4-43f1-aa72-ad54957fd942 admin admin] PATH_INFO: `/v3/projects/ceb3e235e0a3417da5eec6a6fae8b102` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:07:11.732443 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431017b6-f8d4-43f1-aa72-ad54957fd942 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceb3e235e0a3417da5eec6a6fae8b102)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:07:11.734348 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431017b6-f8d4-43f1-aa72-ad54957fd942 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:07:11.780187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-431017b6-f8d4-43f1-aa72-ad54957fd942 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceb3e235e0a3417da5eec6a6fae8b102', 'request_id': 'req-431017b6-f8d4-43f1-aa72-ad54957fd942'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:11.787086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-431017b6-f8d4-43f1-aa72-ad54957fd942 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceb3e235e0a3417da5eec6a6fae8b102', 'request_id': 'req-431017b6-f8d4-43f1-aa72-ad54957fd942'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:11.793367 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-431017b6-f8d4-43f1-aa72-ad54957fd942 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceb3e235e0a3417da5eec6a6fae8b102', 'request_id': 'req-431017b6-f8d4-43f1-aa72-ad54957fd942'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:11.805154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-431017b6-f8d4-43f1-aa72-ad54957fd942 admin admin] The token cache is being invalidate because project ceb3e235e0a3417da5eec6a6fae8b102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:07:11.805439 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-431017b6-f8d4-43f1-aa72-ad54957fd942 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:11.806072 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-431017b6-f8d4-43f1-aa72-ad54957fd942 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:07:11.808274 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1376/2795] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:07:11 2022] DELETE /identity/v3/projects/ceb3e235e0a3417da5eec6a6fae8b102 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:07:21.477590 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:07:21.802174 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:07:22.612520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b100e-5978-4a8e-93b1-8774094512d2 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:07:22.613017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b100e-5978-4a8e-93b1-8774094512d2 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:07:22.613144 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b100e-5978-4a8e-93b1-8774094512d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:07:22.631628 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-419b100e-5978-4a8e-93b1-8774094512d2 None None] MFA Rules not processed for user `5272710cad204a49a52116becbce6bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:07:22.632849 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-419b100e-5978-4a8e-93b1-8774094512d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:07:22.687521 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1420/2796] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:07:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:07:29.018845 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431017b6-f8d4-43f1-aa72-ad54957fd942 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:07:29.020431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-431017b6-f8d4-43f1-aa72-ad54957fd942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:07:29.078914 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431017b6-f8d4-43f1-aa72-ad54957fd942 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:07:29.082595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32032094-4c00-4def-b0eb-5e898a0ba38c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:07:29.083249 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032094-4c00-4def-b0eb-5e898a0ba38c service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:07:29.083530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032094-4c00-4def-b0eb-5e898a0ba38c service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:07:29.083835 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032094-4c00-4def-b0eb-5e898a0ba38c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:07:29.085704 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-32032094-4c00-4def-b0eb-5e898a0ba38c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:07:29.159599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32032094-4c00-4def-b0eb-5e898a0ba38c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:07:29.161088 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32032094-4c00-4def-b0eb-5e898a0ba38c service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:07:29.172440 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1377/2797] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:07:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 21:07:30.352065 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b100e-5978-4a8e-93b1-8774094512d2 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:07:30.353721 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-419b100e-5978-4a8e-93b1-8774094512d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:07:30.366074 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b100e-5978-4a8e-93b1-8774094512d2 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:07:30.370790 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdeea4-4721-43ba-b863-a83e4c71f0c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:07:30.372217 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdeea4-4721-43ba-b863-a83e4c71f0c6 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:07:30.372501 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdeea4-4721-43ba-b863-a83e4c71f0c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:07:30.372778 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdeea4-4721-43ba-b863-a83e4c71f0c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:07:30.374565 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-97bdeea4-4721-43ba-b863-a83e4c71f0c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:07:30.388913 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdeea4-4721-43ba-b863-a83e4c71f0c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:07:30.390341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdeea4-4721-43ba-b863-a83e4c71f0c6 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:07:30.401543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1421/2798] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:07:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:07:39.168187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=54 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:07:40.396493 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=31 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:09:04.477123 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c851924-50cc-41f5-ac97-df3bfedfa1c1 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:09:04.477643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c851924-50cc-41f5-ac97-df3bfedfa1c1 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:09:04.477643 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c851924-50cc-41f5-ac97-df3bfedfa1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:09:04.495617 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-9c851924-50cc-41f5-ac97-df3bfedfa1c1 None None] MFA Rules not processed for user `5272710cad204a49a52116becbce6bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:09:04.496851 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-9c851924-50cc-41f5-ac97-df3bfedfa1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:09:04.504536 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1378/2799] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:09:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:09:04.806266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdeea4-4721-43ba-b863-a83e4c71f0c6 service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:09:04.814945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdeea4-4721-43ba-b863-a83e4c71f0c6 service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:09:04.819267 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584297dd-305b-495a-805c-b9d14e489deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:09:04.819940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584297dd-305b-495a-805c-b9d14e489deb service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:09:04.820225 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584297dd-305b-495a-805c-b9d14e489deb service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:09:04.820503 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584297dd-305b-495a-805c-b9d14e489deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:09:04.822279 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-584297dd-305b-495a-805c-b9d14e489deb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:09:04.832454 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584297dd-305b-495a-805c-b9d14e489deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:09:04.833861 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584297dd-305b-495a-805c-b9d14e489deb service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:09:04.841826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1422/2800] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:09:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:09:06.166979 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c851924-50cc-41f5-ac97-df3bfedfa1c1 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:09:06.168452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-9c851924-50cc-41f5-ac97-df3bfedfa1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:09:06.241123 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c851924-50cc-41f5-ac97-df3bfedfa1c1 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:09:06.246950 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83822126-8b07-4b40-a36e-39c5be41825d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:09:06.247627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83822126-8b07-4b40-a36e-39c5be41825d service placement] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:09:06.247947 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83822126-8b07-4b40-a36e-39c5be41825d service placement] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:09:06.248227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83822126-8b07-4b40-a36e-39c5be41825d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:09:06.250128 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-83822126-8b07-4b40-a36e-39c5be41825d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:09:06.264314 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83822126-8b07-4b40-a36e-39c5be41825d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:09:06.265932 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83822126-8b07-4b40-a36e-39c5be41825d service placement] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:09:06.277143 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1379/2801] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:09:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:09:09.370065 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584297dd-305b-495a-805c-b9d14e489deb service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:09:09.379211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584297dd-305b-495a-805c-b9d14e489deb service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:09:09.382762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7be35a-65de-47bb-879a-ef30275a6969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:09:09.383391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7be35a-65de-47bb-879a-ef30275a6969 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:09:09.383670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7be35a-65de-47bb-879a-ef30275a6969 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:09:09.383979 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7be35a-65de-47bb-879a-ef30275a6969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:09:09.385749 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-bc7be35a-65de-47bb-879a-ef30275a6969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:09:09.395859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7be35a-65de-47bb-879a-ef30275a6969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:09:09.397288 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7be35a-65de-47bb-879a-ef30275a6969 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:09:09.405788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1423/2802] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:09:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:09:12.777024 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83822126-8b07-4b40-a36e-39c5be41825d service placement] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:09:12.778574 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-83822126-8b07-4b40-a36e-39c5be41825d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:09:12.798141 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83822126-8b07-4b40-a36e-39c5be41825d service placement] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:09:12.805406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebb626-b407-43fb-b940-0f44a741160b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:09:12.806193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebb626-b407-43fb-b940-0f44a741160b service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:09:12.806579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebb626-b407-43fb-b940-0f44a741160b service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:09:12.806961 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebb626-b407-43fb-b940-0f44a741160b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:09:12.809039 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f7ebb626-b407-43fb-b940-0f44a741160b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:09:12.823413 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebb626-b407-43fb-b940-0f44a741160b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:09:12.825244 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebb626-b407-43fb-b940-0f44a741160b service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:09:12.838516 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1380/2803] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:09:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:09:12.842346 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba01bd-dd2c-489c-a7de-2febdc8b4cde None None] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:09:12.842848 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba01bd-dd2c-489c-a7de-2febdc8b4cde None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:09:12.842979 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba01bd-dd2c-489c-a7de-2febdc8b4cde None None] PATH_INFO: `/` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:09:12.844479 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1424/2804] 10.222.0.60 () {54 vars in 921 bytes} [Thu Jul 21 21:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 21:09:12.852417 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07087cc7-40dc-432c-a59d-5a3bf83ba3e8 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:09:12.852703 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07087cc7-40dc-432c-a59d-5a3bf83ba3e8 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:09:12.853050 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07087cc7-40dc-432c-a59d-5a3bf83ba3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:09:12.923257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-07087cc7-40dc-432c-a59d-5a3bf83ba3e8 None None] MFA Rules not processed for user `33aff48933844401b86eb808937e35c4`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:09:12.925373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-07087cc7-40dc-432c-a59d-5a3bf83ba3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:09:12.941006 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1381/2805] 10.222.0.60 () {58 vars in 1018 bytes} [Thu Jul 21 21:09:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 21:09:12.970906 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba01bd-dd2c-489c-a7de-2febdc8b4cde None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:09:12.985509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba01bd-dd2c-489c-a7de-2febdc8b4cde None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:09:12.990269 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f0f33-ee11-4829-a9c1-92a6f7a0a233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:09:12.990925 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f0f33-ee11-4829-a9c1-92a6f7a0a233 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:09:12.991215 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f0f33-ee11-4829-a9c1-92a6f7a0a233 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:09:12.991490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f0f33-ee11-4829-a9c1-92a6f7a0a233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:09:13.005721 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f0f33-ee11-4829-a9c1-92a6f7a0a233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:09:13.007512 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f0f33-ee11-4829-a9c1-92a6f7a0a233 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:09:13.017689 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1425/2806] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:09:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:09:13.109938 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2cbca-12ef-4cfa-bf43-e965ee6c5a31 None None] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:09:13.110224 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2cbca-12ef-4cfa-bf43-e965ee6c5a31 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:09:13.110687 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2cbca-12ef-4cfa-bf43-e965ee6c5a31 None None] PATH_INFO: `/` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:09:13.112328 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1382/2807] 10.222.0.60 () {54 vars in 921 bytes} [Thu Jul 21 21:09:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 21:09:13.119055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626366b4-e88d-44e0-9e5f-d21f1b18911f None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:09:13.119508 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626366b4-e88d-44e0-9e5f-d21f1b18911f None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:09:13.119910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626366b4-e88d-44e0-9e5f-d21f1b18911f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:09:13.160004 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-626366b4-e88d-44e0-9e5f-d21f1b18911f None None] MFA Rules not processed for user `77812b356c974d2b80102217956169db`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:09:13.161223 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-626366b4-e88d-44e0-9e5f-d21f1b18911f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:09:13.171664 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1426/2808] 10.222.0.60 () {58 vars in 1018 bytes} [Thu Jul 21 21:09:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:09:13.183303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2cbca-12ef-4cfa-bf43-e965ee6c5a31 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:09:13.195495 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2cbca-12ef-4cfa-bf43-e965ee6c5a31 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:09:13.199300 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe196d2-a196-4aae-aa94-33852bd2db89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:09:13.199999 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe196d2-a196-4aae-aa94-33852bd2db89 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:09:13.200315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe196d2-a196-4aae-aa94-33852bd2db89 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:09:13.200604 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe196d2-a196-4aae-aa94-33852bd2db89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:09:13.212145 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe196d2-a196-4aae-aa94-33852bd2db89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:09:13.213583 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe196d2-a196-4aae-aa94-33852bd2db89 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:09:13.225675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1383/2809] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:09:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:09:14.631070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626366b4-e88d-44e0-9e5f-d21f1b18911f None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:09:14.632721 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-626366b4-e88d-44e0-9e5f-d21f1b18911f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:09:14.648804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626366b4-e88d-44e0-9e5f-d21f1b18911f None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:09:14.655077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d451a22-eab6-411e-8023-ddc55eaaec50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:09:14.655875 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d451a22-eab6-411e-8023-ddc55eaaec50 service nova] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:09:14.656261 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d451a22-eab6-411e-8023-ddc55eaaec50 service nova] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:09:14.656641 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d451a22-eab6-411e-8023-ddc55eaaec50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:09:14.658730 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1d451a22-eab6-411e-8023-ddc55eaaec50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:09:14.672518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d451a22-eab6-411e-8023-ddc55eaaec50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:09:14.674280 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d451a22-eab6-411e-8023-ddc55eaaec50 service nova] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:09:14.686863 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1427/2810] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:09:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:09:23.219875 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=56 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:09:24.682169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=33 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:10:38.846779 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe196d2-a196-4aae-aa94-33852bd2db89 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:10:38.915130 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe196d2-a196-4aae-aa94-33852bd2db89 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:10:38.920755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997d653-9768-4b5b-b8b6-98f201d22f47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77812b356c974d2b80102217956169db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:10:38.921514 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997d653-9768-4b5b-b8b6-98f201d22f47 service nova] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:10:38.921796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997d653-9768-4b5b-b8b6-98f201d22f47 service nova] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:10:38.922073 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997d653-9768-4b5b-b8b6-98f201d22f47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:10:38.923871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-5997d653-9768-4b5b-b8b6-98f201d22f47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:10:38.943867 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997d653-9768-4b5b-b8b6-98f201d22f47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:10:38.945638 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997d653-9768-4b5b-b8b6-98f201d22f47 service nova] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:10:38.956690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1384/2811] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:10:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:10:48.954091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=34 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:10:55.108303 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfa3bb-9107-46d6-8d1b-44d315d706b2 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:10:55.108829 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfa3bb-9107-46d6-8d1b-44d315d706b2 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:10:55.108932 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfa3bb-9107-46d6-8d1b-44d315d706b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:10:55.127975 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-1ddfa3bb-9107-46d6-8d1b-44d315d706b2 None None] MFA Rules not processed for user `5272710cad204a49a52116becbce6bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:10:55.129186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-1ddfa3bb-9107-46d6-8d1b-44d315d706b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:10:55.136223 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1428/2812] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:10:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:11:05.121765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:11:13.145633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997d653-9768-4b5b-b8b6-98f201d22f47 service nova] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:11:13.164948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997d653-9768-4b5b-b8b6-98f201d22f47 service nova] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:11:13.169301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c55b6-71eb-4b4e-809e-be06804ea9c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:11:13.169941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c55b6-71eb-4b4e-809e-be06804ea9c1 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:11:13.170218 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c55b6-71eb-4b4e-809e-be06804ea9c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:11:13.170489 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c55b6-71eb-4b4e-809e-be06804ea9c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:11:13.172277 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-472c55b6-71eb-4b4e-809e-be06804ea9c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:11:13.232200 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c55b6-71eb-4b4e-809e-be06804ea9c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:11:13.233558 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c55b6-71eb-4b4e-809e-be06804ea9c1 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:11:13.244013 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1385/2813] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:11:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:11:17.062586 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfa3bb-9107-46d6-8d1b-44d315d706b2 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:11:17.084619 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfa3bb-9107-46d6-8d1b-44d315d706b2 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:11:17.090406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1ec15-5bfa-4380-9667-710bb0c2d1a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:11:17.091048 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1ec15-5bfa-4380-9667-710bb0c2d1a6 service placement] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:11:17.091331 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1ec15-5bfa-4380-9667-710bb0c2d1a6 service placement] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:11:17.091606 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1ec15-5bfa-4380-9667-710bb0c2d1a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:11:17.093428 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-83a1ec15-5bfa-4380-9667-710bb0c2d1a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:11:17.125064 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a1ec15-5bfa-4380-9667-710bb0c2d1a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:11:17.126659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a1ec15-5bfa-4380-9667-710bb0c2d1a6 service placement] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:11:17.137610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1429/2814] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:11:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:11:17.939339 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c55b6-71eb-4b4e-809e-be06804ea9c1 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:11:17.949098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c55b6-71eb-4b4e-809e-be06804ea9c1 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:11:17.952826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c785e1-1f94-4110-b5e0-ad38fe560136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:11:17.953475 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c785e1-1f94-4110-b5e0-ad38fe560136 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:11:17.953764 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c785e1-1f94-4110-b5e0-ad38fe560136 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:11:17.954168 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c785e1-1f94-4110-b5e0-ad38fe560136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:11:17.956010 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b5c785e1-1f94-4110-b5e0-ad38fe560136 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:11:18.023653 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c785e1-1f94-4110-b5e0-ad38fe560136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:11:18.025836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c785e1-1f94-4110-b5e0-ad38fe560136 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:11:18.037184 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1386/2815] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:11:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:11:24.307467 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1ec15-5bfa-4380-9667-710bb0c2d1a6 service placement] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:11:24.318252 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-83a1ec15-5bfa-4380-9667-710bb0c2d1a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:11:24.331904 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1ec15-5bfa-4380-9667-710bb0c2d1a6 service placement] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:11:24.336486 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9b549-5063-4371-8350-c62d8cc22f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:11:24.337966 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9b549-5063-4371-8350-c62d8cc22f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:11:24.338254 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9b549-5063-4371-8350-c62d8cc22f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:11:24.338529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9b549-5063-4371-8350-c62d8cc22f16 admin admin] PATH_INFO: `/v3/users/f06aae5f002d46a3a60100c908d07455` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:11:24.339759 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9b549-5063-4371-8350-c62d8cc22f16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f06aae5f002d46a3a60100c908d07455)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:11:24.341625 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9b549-5063-4371-8350-c62d8cc22f16 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:11:24.388045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-28a9b549-5063-4371-8350-c62d8cc22f16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f06aae5f002d46a3a60100c908d07455'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:24.388707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-28a9b549-5063-4371-8350-c62d8cc22f16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f06aae5f002d46a3a60100c908d07455'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:24.401395 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-28a9b549-5063-4371-8350-c62d8cc22f16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f06aae5f002d46a3a60100c908d07455'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:24.408710 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-28a9b549-5063-4371-8350-c62d8cc22f16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f06aae5f002d46a3a60100c908d07455'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:24.417415 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-28a9b549-5063-4371-8350-c62d8cc22f16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f06aae5f002d46a3a60100c908d07455'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:24.423458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1430/2816] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:11:24 2022] DELETE /identity/v3/users/f06aae5f002d46a3a60100c908d07455 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:11:24.594808 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c785e1-1f94-4110-b5e0-ad38fe560136 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:11:24.606837 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c785e1-1f94-4110-b5e0-ad38fe560136 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:11:24.663047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036dd81-fe3f-4742-9842-fddbeafeb679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:11:24.663946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036dd81-fe3f-4742-9842-fddbeafeb679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:11:24.664230 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036dd81-fe3f-4742-9842-fddbeafeb679 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:11:24.664508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036dd81-fe3f-4742-9842-fddbeafeb679 admin admin] PATH_INFO: `/v3/projects/91eaf4caa6ea457bafa7de7a1387851c` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:11:24.666333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036dd81-fe3f-4742-9842-fddbeafeb679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91eaf4caa6ea457bafa7de7a1387851c)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:11:24.668272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036dd81-fe3f-4742-9842-fddbeafeb679 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:11:24.699507 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:11:24.716438 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-3036dd81-fe3f-4742-9842-fddbeafeb679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91eaf4caa6ea457bafa7de7a1387851c', 'request_id': 'req-3036dd81-fe3f-4742-9842-fddbeafeb679'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:24.723870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-3036dd81-fe3f-4742-9842-fddbeafeb679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91eaf4caa6ea457bafa7de7a1387851c', 'request_id': 'req-3036dd81-fe3f-4742-9842-fddbeafeb679'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:24.729551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-3036dd81-fe3f-4742-9842-fddbeafeb679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91eaf4caa6ea457bafa7de7a1387851c', 'request_id': 'req-3036dd81-fe3f-4742-9842-fddbeafeb679'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:24.740531 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-3036dd81-fe3f-4742-9842-fddbeafeb679 admin admin] The token cache is being invalidate because project 91eaf4caa6ea457bafa7de7a1387851c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:11:24.740816 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-3036dd81-fe3f-4742-9842-fddbeafeb679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:24.741452 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-3036dd81-fe3f-4742-9842-fddbeafeb679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:24.743058 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1387/2817] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:11:24 2022] DELETE /identity/v3/projects/91eaf4caa6ea457bafa7de7a1387851c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:11:24.747527 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9b549-5063-4371-8350-c62d8cc22f16 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:11:24.748959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-28a9b549-5063-4371-8350-c62d8cc22f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:11:24.762787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9b549-5063-4371-8350-c62d8cc22f16 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:11:24.818931 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2dad7-9a17-4aed-8c38-cb79783eecd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:11:24.819829 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2dad7-9a17-4aed-8c38-cb79783eecd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:11:24.820116 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2dad7-9a17-4aed-8c38-cb79783eecd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:11:24.820391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2dad7-9a17-4aed-8c38-cb79783eecd5 admin admin] PATH_INFO: `/v3/users/cb292a64c14d4e66b974e9f9d1022d66` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:11:24.822358 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2dad7-9a17-4aed-8c38-cb79783eecd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb292a64c14d4e66b974e9f9d1022d66)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:11:24.824310 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2dad7-9a17-4aed-8c38-cb79783eecd5 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:11:24.863012 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:11:24.868743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b2a2dad7-9a17-4aed-8c38-cb79783eecd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb292a64c14d4e66b974e9f9d1022d66'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:24.870328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b2a2dad7-9a17-4aed-8c38-cb79783eecd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb292a64c14d4e66b974e9f9d1022d66'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:24.882948 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b2a2dad7-9a17-4aed-8c38-cb79783eecd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb292a64c14d4e66b974e9f9d1022d66'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:24.890668 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b2a2dad7-9a17-4aed-8c38-cb79783eecd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb292a64c14d4e66b974e9f9d1022d66'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:24.897539 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-b2a2dad7-9a17-4aed-8c38-cb79783eecd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb292a64c14d4e66b974e9f9d1022d66'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:24.903434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1431/2818] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:11:24 2022] DELETE /identity/v3/users/cb292a64c14d4e66b974e9f9d1022d66 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:11:25.088934 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036dd81-fe3f-4742-9842-fddbeafeb679 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:11:25.090423 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-3036dd81-fe3f-4742-9842-fddbeafeb679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:11:25.104778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036dd81-fe3f-4742-9842-fddbeafeb679 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:11:25.162248 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd8c96-adda-4f4d-a228-2626e4aa4754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:11:25.163107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd8c96-adda-4f4d-a228-2626e4aa4754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:11:25.163390 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd8c96-adda-4f4d-a228-2626e4aa4754 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:11:25.163665 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd8c96-adda-4f4d-a228-2626e4aa4754 admin admin] PATH_INFO: `/v3/projects/4bd5a82e19f0453fa0585af78fc35e6a` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:11:25.165458 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd8c96-adda-4f4d-a228-2626e4aa4754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd5a82e19f0453fa0585af78fc35e6a)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:11:25.168300 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd8c96-adda-4f4d-a228-2626e4aa4754 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:11:25.214755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-13cd8c96-adda-4f4d-a228-2626e4aa4754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd5a82e19f0453fa0585af78fc35e6a', 'request_id': 'req-13cd8c96-adda-4f4d-a228-2626e4aa4754'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:25.221874 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-13cd8c96-adda-4f4d-a228-2626e4aa4754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd5a82e19f0453fa0585af78fc35e6a', 'request_id': 'req-13cd8c96-adda-4f4d-a228-2626e4aa4754'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:25.227486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-13cd8c96-adda-4f4d-a228-2626e4aa4754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd5a82e19f0453fa0585af78fc35e6a', 'request_id': 'req-13cd8c96-adda-4f4d-a228-2626e4aa4754'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:25.238579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-13cd8c96-adda-4f4d-a228-2626e4aa4754 admin admin] The token cache is being invalidate because project 4bd5a82e19f0453fa0585af78fc35e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:11:25.238862 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-13cd8c96-adda-4f4d-a228-2626e4aa4754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:25.239490 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-13cd8c96-adda-4f4d-a228-2626e4aa4754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:11:25.241076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1388/2819] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:11:25 2022] DELETE /identity/v3/projects/4bd5a82e19f0453fa0585af78fc35e6a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:11:34.899877 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=14,keystone:DELETE=4,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:11:35.235574 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:DELETE=13,keystone:SELECT=65,keystone:INSERT=2 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:31.574997 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2dad7-9a17-4aed-8c38-cb79783eecd5 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:31.576424 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b2a2dad7-9a17-4aed-8c38-cb79783eecd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:31.656555 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2dad7-9a17-4aed-8c38-cb79783eecd5 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:31.706721 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef23010-18d0-4b98-9471-8e8587c4e47f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:31.707373 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef23010-18d0-4b98-9471-8e8587c4e47f service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:31.707659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef23010-18d0-4b98-9471-8e8587c4e47f service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:31.707968 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef23010-18d0-4b98-9471-8e8587c4e47f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:31.709765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8ef23010-18d0-4b98-9471-8e8587c4e47f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:31.793887 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef23010-18d0-4b98-9471-8e8587c4e47f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:31.795405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef23010-18d0-4b98-9471-8e8587c4e47f service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:31.806680 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1432/2820] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:12:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:12:34.429236 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd8c96-adda-4f4d-a228-2626e4aa4754 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:34.430664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-13cd8c96-adda-4f4d-a228-2626e4aa4754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:34.506733 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd8c96-adda-4f4d-a228-2626e4aa4754 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:34.510424 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3954fc-8f3a-49a9-bb31-d4a6eaa1925b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:34.511072 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3954fc-8f3a-49a9-bb31-d4a6eaa1925b service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:34.511394 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3954fc-8f3a-49a9-bb31-d4a6eaa1925b service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:34.511675 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3954fc-8f3a-49a9-bb31-d4a6eaa1925b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:34.516578 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8c3954fc-8f3a-49a9-bb31-d4a6eaa1925b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:34.528803 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3954fc-8f3a-49a9-bb31-d4a6eaa1925b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:34.530238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3954fc-8f3a-49a9-bb31-d4a6eaa1925b service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:34.541519 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1389/2821] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:12:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:12:41.107958 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef23010-18d0-4b98-9471-8e8587c4e47f service cinder] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:41.109361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-8ef23010-18d0-4b98-9471-8e8587c4e47f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:41.121488 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef23010-18d0-4b98-9471-8e8587c4e47f service cinder] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:41.125533 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad0056-6b40-4dc2-8b17-18ad1fa7532c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:41.126191 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad0056-6b40-4dc2-8b17-18ad1fa7532c service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:41.126473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad0056-6b40-4dc2-8b17-18ad1fa7532c service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:41.126748 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad0056-6b40-4dc2-8b17-18ad1fa7532c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:41.128540 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f8ad0056-6b40-4dc2-8b17-18ad1fa7532c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:41.156076 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=100 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:41.213706 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad0056-6b40-4dc2-8b17-18ad1fa7532c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:41.215350 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad0056-6b40-4dc2-8b17-18ad1fa7532c service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:41.234240 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1433/2822] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:12:44.296082 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3954fc-8f3a-49a9-bb31-d4a6eaa1925b service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:44.297508 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-8c3954fc-8f3a-49a9-bb31-d4a6eaa1925b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:44.359860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad0056-6b40-4dc2-8b17-18ad1fa7532c service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:44.361337 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-f8ad0056-6b40-4dc2-8b17-18ad1fa7532c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:44.376948 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3954fc-8f3a-49a9-bb31-d4a6eaa1925b service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:44.382239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d248cf-037b-48da-ba31-27de80b19b70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:44.382547 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad0056-6b40-4dc2-8b17-18ad1fa7532c service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:44.383015 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d248cf-037b-48da-ba31-27de80b19b70 service placement] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:44.383303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d248cf-037b-48da-ba31-27de80b19b70 service placement] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:44.383579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d248cf-037b-48da-ba31-27de80b19b70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:44.385398 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-55d248cf-037b-48da-ba31-27de80b19b70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:44.388502 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11651500-4ce2-4b77-b140-8e0ea0a66708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:44.389371 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651500-4ce2-4b77-b140-8e0ea0a66708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:44.389653 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651500-4ce2-4b77-b140-8e0ea0a66708 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:44.389922 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651500-4ce2-4b77-b140-8e0ea0a66708 admin admin] PATH_INFO: `/v3/users/5272710cad204a49a52116becbce6bc1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:44.391099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11651500-4ce2-4b77-b140-8e0ea0a66708 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5272710cad204a49a52116becbce6bc1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:44.393127 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11651500-4ce2-4b77-b140-8e0ea0a66708 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:44.398319 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d248cf-037b-48da-ba31-27de80b19b70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:44.400023 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d248cf-037b-48da-ba31-27de80b19b70 service placement] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:44.413472 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1390/2823] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:12:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:12:44.447420 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-11651500-4ce2-4b77-b140-8e0ea0a66708 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5272710cad204a49a52116becbce6bc1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:44.448132 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-11651500-4ce2-4b77-b140-8e0ea0a66708 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5272710cad204a49a52116becbce6bc1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:44.460072 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-11651500-4ce2-4b77-b140-8e0ea0a66708 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5272710cad204a49a52116becbce6bc1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:44.468396 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-11651500-4ce2-4b77-b140-8e0ea0a66708 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5272710cad204a49a52116becbce6bc1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:44.475361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-11651500-4ce2-4b77-b140-8e0ea0a66708 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5272710cad204a49a52116becbce6bc1'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:44.481993 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1434/2824] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:12:44 2022] DELETE /identity/v3/users/5272710cad204a49a52116becbce6bc1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:12:44.666111 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d248cf-037b-48da-ba31-27de80b19b70 service placement] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:44.677858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d248cf-037b-48da-ba31-27de80b19b70 service placement] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:44.735760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39b798-566f-4ea4-b9ff-4364cba5e567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:44.736624 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39b798-566f-4ea4-b9ff-4364cba5e567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:44.736908 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39b798-566f-4ea4-b9ff-4364cba5e567 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:44.737186 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39b798-566f-4ea4-b9ff-4364cba5e567 admin admin] PATH_INFO: `/v3/projects/6549bea44b794d03bab57a0c2a001992` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:44.738449 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a39b798-566f-4ea4-b9ff-4364cba5e567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6549bea44b794d03bab57a0c2a001992)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:44.740465 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a39b798-566f-4ea4-b9ff-4364cba5e567 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:44.765392 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:44.788865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9a39b798-566f-4ea4-b9ff-4364cba5e567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6549bea44b794d03bab57a0c2a001992', 'request_id': 'req-9a39b798-566f-4ea4-b9ff-4364cba5e567'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:44.796230 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9a39b798-566f-4ea4-b9ff-4364cba5e567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6549bea44b794d03bab57a0c2a001992', 'request_id': 'req-9a39b798-566f-4ea4-b9ff-4364cba5e567'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:44.802065 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9a39b798-566f-4ea4-b9ff-4364cba5e567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6549bea44b794d03bab57a0c2a001992', 'request_id': 'req-9a39b798-566f-4ea4-b9ff-4364cba5e567'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:44.813082 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9a39b798-566f-4ea4-b9ff-4364cba5e567 admin admin] The token cache is being invalidate because project 6549bea44b794d03bab57a0c2a001992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:12:44.813373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9a39b798-566f-4ea4-b9ff-4364cba5e567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:44.814520 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9a39b798-566f-4ea4-b9ff-4364cba5e567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:44.819915 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1391/2825] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:12:44 2022] DELETE /identity/v3/projects/6549bea44b794d03bab57a0c2a001992 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:12:44.825822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72064c23-6658-4ec7-9462-9160d775b429 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:44.826118 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72064c23-6658-4ec7-9462-9160d775b429 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:44.826400 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72064c23-6658-4ec7-9462-9160d775b429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:44.885204 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-72064c23-6658-4ec7-9462-9160d775b429 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:12:44.886939 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-72064c23-6658-4ec7-9462-9160d775b429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:44.914959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:44.947464 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1435/2826] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:12:44.955967 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35188386-270d-4f80-a7cb-d55865d3df78 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:44.956257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35188386-270d-4f80-a7cb-d55865d3df78 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:44.956542 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35188386-270d-4f80-a7cb-d55865d3df78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:44.975925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-35188386-270d-4f80-a7cb-d55865d3df78 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:12:44.977146 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-35188386-270d-4f80-a7cb-d55865d3df78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:44.985454 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1392/2827] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:12:44.989722 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72064c23-6658-4ec7-9462-9160d775b429 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:45.000535 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72064c23-6658-4ec7-9462-9160d775b429 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:45.005518 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eab73f-eae2-4281-8429-cb3200f6d267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:45.008585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eab73f-eae2-4281-8429-cb3200f6d267 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:45.008967 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eab73f-eae2-4281-8429-cb3200f6d267 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:45.009311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eab73f-eae2-4281-8429-cb3200f6d267 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:45.010135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eab73f-eae2-4281-8429-cb3200f6d267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:12:45.010507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eab73f-eae2-4281-8429-cb3200f6d267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:45.013031 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eab73f-eae2-4281-8429-cb3200f6d267 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:45.025284 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1436/2828] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:12:45.029850 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35188386-270d-4f80-a7cb-d55865d3df78 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:45.039491 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35188386-270d-4f80-a7cb-d55865d3df78 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:45.044566 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c02ff-b55f-4ccb-a65a-e0b812f3edf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:45.045235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c02ff-b55f-4ccb-a65a-e0b812f3edf3 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:45.045519 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c02ff-b55f-4ccb-a65a-e0b812f3edf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:45.045792 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c02ff-b55f-4ccb-a65a-e0b812f3edf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:45.046497 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c02ff-b55f-4ccb-a65a-e0b812f3edf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:45.048446 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c02ff-b55f-4ccb-a65a-e0b812f3edf3 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:45.076241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1393/2829] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:12:45 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:12:45.080408 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eab73f-eae2-4281-8429-cb3200f6d267 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:45.090520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eab73f-eae2-4281-8429-cb3200f6d267 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:45.150029 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9d05b-43e3-415a-8ad3-6258e3d7144d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:45.150699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9d05b-43e3-415a-8ad3-6258e3d7144d admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:45.150987 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9d05b-43e3-415a-8ad3-6258e3d7144d admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:45.151266 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9d05b-43e3-415a-8ad3-6258e3d7144d admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:45.152585 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9d05b-43e3-415a-8ad3-6258e3d7144d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:45.154534 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9d05b-43e3-415a-8ad3-6258e3d7144d admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:45.174311 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1437/2830] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:12:45 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:12:45.178438 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c02ff-b55f-4ccb-a65a-e0b812f3edf3 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:45.189987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c02ff-b55f-4ccb-a65a-e0b812f3edf3 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:45.196927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443592c2-6f88-4cea-bb44-12e7f9a680c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:45.197595 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443592c2-6f88-4cea-bb44-12e7f9a680c6 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:45.197890 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443592c2-6f88-4cea-bb44-12e7f9a680c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:45.198213 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443592c2-6f88-4cea-bb44-12e7f9a680c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:45.199098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443592c2-6f88-4cea-bb44-12e7f9a680c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:45.201926 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443592c2-6f88-4cea-bb44-12e7f9a680c6 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:45.339025 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1394/2831] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:12:45.343780 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9d05b-43e3-415a-8ad3-6258e3d7144d admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:45.353791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9d05b-43e3-415a-8ad3-6258e3d7144d admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:45.358570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df2115-0f7f-403c-8673-c04bfbdf514a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:45.359205 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df2115-0f7f-403c-8673-c04bfbdf514a admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:45.359484 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df2115-0f7f-403c-8673-c04bfbdf514a admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:45.359797 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df2115-0f7f-403c-8673-c04bfbdf514a admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:45.360505 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df2115-0f7f-403c-8673-c04bfbdf514a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:45.362081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df2115-0f7f-403c-8673-c04bfbdf514a admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:45.372305 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1438/2832] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 21:12:45.376427 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443592c2-6f88-4cea-bb44-12e7f9a680c6 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:45.387156 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443592c2-6f88-4cea-bb44-12e7f9a680c6 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:45.391955 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79a878-3802-4d43-9d59-aa0ffa00849a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:45.394002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79a878-3802-4d43-9d59-aa0ffa00849a admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:45.394002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79a878-3802-4d43-9d59-aa0ffa00849a admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:45.394002 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79a878-3802-4d43-9d59-aa0ffa00849a admin admin] PATH_INFO: `/v3/projects/9eb51b1a88224365a114d672419eae21/users/2a8446da78904f648968dbca56a33cb9/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:45.409312 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79a878-3802-4d43-9d59-aa0ffa00849a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb51b1a88224365a114d672419eae21, user_id=2a8446da78904f648968dbca56a33cb9, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:45.411969 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79a878-3802-4d43-9d59-aa0ffa00849a admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:45.424872 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1395/2833] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:12:45 2022] PUT /identity/v3/projects/9eb51b1a88224365a114d672419eae21/users/2a8446da78904f648968dbca56a33cb9/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:12:45.435606 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268a9e1-0c97-4009-9b76-82fa0bd71d42 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:45.435926 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268a9e1-0c97-4009-9b76-82fa0bd71d42 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:45.436231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268a9e1-0c97-4009-9b76-82fa0bd71d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:45.462977 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-d268a9e1-0c97-4009-9b76-82fa0bd71d42 None None] MFA Rules not processed for user `2a8446da78904f648968dbca56a33cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:12:45.464244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d268a9e1-0c97-4009-9b76-82fa0bd71d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:45.526881 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1439/2834] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:12:45 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:12:45.538693 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79a878-3802-4d43-9d59-aa0ffa00849a admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:45.540431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-7c79a878-3802-4d43-9d59-aa0ffa00849a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:45.598547 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79a878-3802-4d43-9d59-aa0ffa00849a admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:45.602950 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cce66-69f1-4d7a-8fa1-dc22d744aa3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:45.603759 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cce66-69f1-4d7a-8fa1-dc22d744aa3c service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:45.604042 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cce66-69f1-4d7a-8fa1-dc22d744aa3c service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:45.604316 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cce66-69f1-4d7a-8fa1-dc22d744aa3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:45.615166 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9cce66-69f1-4d7a-8fa1-dc22d744aa3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:45.616741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9cce66-69f1-4d7a-8fa1-dc22d744aa3c service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:45.626589 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1396/2835] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:12:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:12:53.874437 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268a9e1-0c97-4009-9b76-82fa0bd71d42 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:53.905045 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:53.937174 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268a9e1-0c97-4009-9b76-82fa0bd71d42 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:53.942105 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c4d52-ef21-4c7a-b2d3-3053ef8c193c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:53.942976 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c4d52-ef21-4c7a-b2d3-3053ef8c193c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:53.943257 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c4d52-ef21-4c7a-b2d3-3053ef8c193c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:53.943532 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c4d52-ef21-4c7a-b2d3-3053ef8c193c admin admin] PATH_INFO: `/v3/users/2a8446da78904f648968dbca56a33cb9` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:53.944782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c4d52-ef21-4c7a-b2d3-3053ef8c193c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a8446da78904f648968dbca56a33cb9)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:53.946699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c4d52-ef21-4c7a-b2d3-3053ef8c193c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:53.992532 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-773c4d52-ef21-4c7a-b2d3-3053ef8c193c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a8446da78904f648968dbca56a33cb9'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:53.993136 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-773c4d52-ef21-4c7a-b2d3-3053ef8c193c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a8446da78904f648968dbca56a33cb9'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:54.005398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-773c4d52-ef21-4c7a-b2d3-3053ef8c193c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a8446da78904f648968dbca56a33cb9'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:54.012473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-773c4d52-ef21-4c7a-b2d3-3053ef8c193c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a8446da78904f648968dbca56a33cb9'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:54.019529 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-773c4d52-ef21-4c7a-b2d3-3053ef8c193c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a8446da78904f648968dbca56a33cb9'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:54.025346 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1440/2836] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:12:53 2022] DELETE /identity/v3/users/2a8446da78904f648968dbca56a33cb9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:12:54.030563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf9f15-633b-4fdd-9ea8-7010a028f0d4 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:54.031055 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf9f15-633b-4fdd-9ea8-7010a028f0d4 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:54.031171 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf9f15-633b-4fdd-9ea8-7010a028f0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:54.048936 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-0eaf9f15-633b-4fdd-9ea8-7010a028f0d4 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:12:54.050122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-0eaf9f15-633b-4fdd-9ea8-7010a028f0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:54.094241 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:54.110716 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1397/2837] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:12:54.120720 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c4d52-ef21-4c7a-b2d3-3053ef8c193c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:54.122770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-773c4d52-ef21-4c7a-b2d3-3053ef8c193c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:54.178433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c4d52-ef21-4c7a-b2d3-3053ef8c193c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:54.182092 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ba11a-da70-4652-808f-1c9faca372c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:54.182730 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ba11a-da70-4652-808f-1c9faca372c4 service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:54.183017 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ba11a-da70-4652-808f-1c9faca372c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:54.183298 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ba11a-da70-4652-808f-1c9faca372c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:54.192980 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ba11a-da70-4652-808f-1c9faca372c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:54.194405 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ba11a-da70-4652-808f-1c9faca372c4 service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:54.204634 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1441/2838] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:12:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:12:54.474832 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf9f15-633b-4fdd-9ea8-7010a028f0d4 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:54.476420 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-0eaf9f15-633b-4fdd-9ea8-7010a028f0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:54.490826 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf9f15-633b-4fdd-9ea8-7010a028f0d4 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:54.495891 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a276a42-82c5-47db-9d49-466ff316433c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:54.496751 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a276a42-82c5-47db-9d49-466ff316433c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:54.497672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a276a42-82c5-47db-9d49-466ff316433c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:54.498135 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a276a42-82c5-47db-9d49-466ff316433c admin admin] PATH_INFO: `/v3/projects/9eb51b1a88224365a114d672419eae21` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:54.499461 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a276a42-82c5-47db-9d49-466ff316433c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb51b1a88224365a114d672419eae21)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:54.501461 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a276a42-82c5-47db-9d49-466ff316433c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:54.548154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6a276a42-82c5-47db-9d49-466ff316433c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb51b1a88224365a114d672419eae21', 'request_id': 'req-6a276a42-82c5-47db-9d49-466ff316433c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:54.555798 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6a276a42-82c5-47db-9d49-466ff316433c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb51b1a88224365a114d672419eae21', 'request_id': 'req-6a276a42-82c5-47db-9d49-466ff316433c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:54.561870 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6a276a42-82c5-47db-9d49-466ff316433c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb51b1a88224365a114d672419eae21', 'request_id': 'req-6a276a42-82c5-47db-9d49-466ff316433c'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:54.572702 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6a276a42-82c5-47db-9d49-466ff316433c admin admin] The token cache is being invalidate because project 9eb51b1a88224365a114d672419eae21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:12:54.572995 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6a276a42-82c5-47db-9d49-466ff316433c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:54.573636 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6a276a42-82c5-47db-9d49-466ff316433c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:54.575155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1398/2839] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:12:54 2022] DELETE /identity/v3/projects/9eb51b1a88224365a114d672419eae21 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:12:54.580570 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6fb1a-0506-4cc5-99ce-673b73975608 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:54.580880 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6fb1a-0506-4cc5-99ce-673b73975608 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:54.581164 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6fb1a-0506-4cc5-99ce-673b73975608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:54.598915 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-9cf6fb1a-0506-4cc5-99ce-673b73975608 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:12:54.600169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9cf6fb1a-0506-4cc5-99ce-673b73975608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:54.646009 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:54.663275 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1442/2840] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:12:54.671785 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efcef9-3717-4d35-923d-26366cdd3d82 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:54.672077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efcef9-3717-4d35-923d-26366cdd3d82 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:54.672356 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efcef9-3717-4d35-923d-26366cdd3d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:54.690177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-07efcef9-3717-4d35-923d-26366cdd3d82 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:12:54.691378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-07efcef9-3717-4d35-923d-26366cdd3d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:54.699344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1399/2841] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:12:54.703398 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6fb1a-0506-4cc5-99ce-673b73975608 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:54.713241 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6fb1a-0506-4cc5-99ce-673b73975608 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:54.718447 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f75d6-b1cc-43ef-9efe-3a5860121cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:54.719117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f75d6-b1cc-43ef-9efe-3a5860121cfc admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:54.719400 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f75d6-b1cc-43ef-9efe-3a5860121cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:54.719677 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f75d6-b1cc-43ef-9efe-3a5860121cfc admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:54.720434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f75d6-b1cc-43ef-9efe-3a5860121cfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:12:54.720740 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f75d6-b1cc-43ef-9efe-3a5860121cfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:54.722327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f75d6-b1cc-43ef-9efe-3a5860121cfc admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:54.734532 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1443/2842] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:12:54.739151 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07efcef9-3717-4d35-923d-26366cdd3d82 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:54.748662 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07efcef9-3717-4d35-923d-26366cdd3d82 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:54.753462 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61e3c9-6bc8-476a-8aae-e9d144884936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:54.754129 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61e3c9-6bc8-476a-8aae-e9d144884936 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:54.754409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61e3c9-6bc8-476a-8aae-e9d144884936 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:54.754682 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61e3c9-6bc8-476a-8aae-e9d144884936 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:54.755362 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61e3c9-6bc8-476a-8aae-e9d144884936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:54.757262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61e3c9-6bc8-476a-8aae-e9d144884936 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:54.782736 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1400/2843] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:12:54 2022] POST /identity/v3/projects => generated 403 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:12:54.786933 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f75d6-b1cc-43ef-9efe-3a5860121cfc admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:54.797743 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f75d6-b1cc-43ef-9efe-3a5860121cfc admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:54.852843 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef790591-ff54-4a53-ae09-98f8b6c8b0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:54.854144 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef790591-ff54-4a53-ae09-98f8b6c8b0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:54.854431 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef790591-ff54-4a53-ae09-98f8b6c8b0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:54.854712 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef790591-ff54-4a53-ae09-98f8b6c8b0c6 admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:54.855402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef790591-ff54-4a53-ae09-98f8b6c8b0c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:54.858070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef790591-ff54-4a53-ae09-98f8b6c8b0c6 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:54.878096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1444/2844] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:12:54 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:12:54.882260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61e3c9-6bc8-476a-8aae-e9d144884936 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:54.892839 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61e3c9-6bc8-476a-8aae-e9d144884936 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:54.897461 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecdd60-8b75-4433-897f-b6905cd0856e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:54.898114 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecdd60-8b75-4433-897f-b6905cd0856e admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:54.898399 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecdd60-8b75-4433-897f-b6905cd0856e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:54.898681 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecdd60-8b75-4433-897f-b6905cd0856e admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:54.899407 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecdd60-8b75-4433-897f-b6905cd0856e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:54.901033 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecdd60-8b75-4433-897f-b6905cd0856e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:54.912043 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1401/2845] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:12:54.915884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef790591-ff54-4a53-ae09-98f8b6c8b0c6 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:54.926341 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef790591-ff54-4a53-ae09-98f8b6c8b0c6 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:54.930927 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb5430-dcfc-40eb-b590-e65980f6b3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:54.931571 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb5430-dcfc-40eb-b590-e65980f6b3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:54.931889 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb5430-dcfc-40eb-b590-e65980f6b3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:54.932166 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb5430-dcfc-40eb-b590-e65980f6b3d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:54.932888 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb5430-dcfc-40eb-b590-e65980f6b3d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:54.934478 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb5430-dcfc-40eb-b590-e65980f6b3d8 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:54.944284 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1445/2846] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:12:54.948152 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecdd60-8b75-4433-897f-b6905cd0856e admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:54.957835 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecdd60-8b75-4433-897f-b6905cd0856e admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:54.962369 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06364425-d696-4a1d-8ff7-8d1ef7ec178e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:54.963227 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06364425-d696-4a1d-8ff7-8d1ef7ec178e admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:54.963858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06364425-d696-4a1d-8ff7-8d1ef7ec178e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:54.963858 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06364425-d696-4a1d-8ff7-8d1ef7ec178e admin admin] PATH_INFO: `/v3/projects/f1f44f5641154e3a82ed5e9c1c31bf83/users/3cb0ebc0a6c24599a32c8affcc7e4c42/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:54.978865 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06364425-d696-4a1d-8ff7-8d1ef7ec178e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1f44f5641154e3a82ed5e9c1c31bf83, user_id=3cb0ebc0a6c24599a32c8affcc7e4c42, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:54.982247 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06364425-d696-4a1d-8ff7-8d1ef7ec178e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:54.995286 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1402/2847] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:12:54 2022] PUT /identity/v3/projects/f1f44f5641154e3a82ed5e9c1c31bf83/users/3cb0ebc0a6c24599a32c8affcc7e4c42/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:12:55.006151 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f793f-25e7-4fc8-9310-429109e81475 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:55.006441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f793f-25e7-4fc8-9310-429109e81475 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:55.006721 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f793f-25e7-4fc8-9310-429109e81475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:55.034549 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-fa3f793f-25e7-4fc8-9310-429109e81475 None None] MFA Rules not processed for user `3cb0ebc0a6c24599a32c8affcc7e4c42`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:12:55.035783 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fa3f793f-25e7-4fc8-9310-429109e81475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:55.099027 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1446/2848] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:12:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:12:55.103618 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06364425-d696-4a1d-8ff7-8d1ef7ec178e admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:55.164203 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06364425-d696-4a1d-8ff7-8d1ef7ec178e admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:55.168995 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c26ad-32f8-4208-89b0-e03649c73e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:55.169871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c26ad-32f8-4208-89b0-e03649c73e48 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:55.170164 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c26ad-32f8-4208-89b0-e03649c73e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:55.170444 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c26ad-32f8-4208-89b0-e03649c73e48 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:55.171146 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c26ad-32f8-4208-89b0-e03649c73e48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:55.173077 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c26ad-32f8-4208-89b0-e03649c73e48 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:55.187184 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:55.203215 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1403/2849] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:12:55 2022] POST /identity/v3/projects => generated 401 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:12:55.204794 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f793f-25e7-4fc8-9310-429109e81475 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:55.215285 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f793f-25e7-4fc8-9310-429109e81475 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:55.260847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:55.275953 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e7c38-ed28-4a7b-a285-4b1c2df0952f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:55.276621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e7c38-ed28-4a7b-a285-4b1c2df0952f admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:55.276902 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e7c38-ed28-4a7b-a285-4b1c2df0952f admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:55.277173 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e7c38-ed28-4a7b-a285-4b1c2df0952f admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:55.277868 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e7c38-ed28-4a7b-a285-4b1c2df0952f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:55.280806 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e7c38-ed28-4a7b-a285-4b1c2df0952f admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:55.304712 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1447/2850] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:12:55 2022] POST /identity/v3/users => generated 307 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:12:55.308836 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c26ad-32f8-4208-89b0-e03649c73e48 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:55.319193 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c26ad-32f8-4208-89b0-e03649c73e48 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:55.323785 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f788b6b-843e-496f-affa-4dabb8b9ee46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:55.324448 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f788b6b-843e-496f-affa-4dabb8b9ee46 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:55.324733 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f788b6b-843e-496f-affa-4dabb8b9ee46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:55.325009 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f788b6b-843e-496f-affa-4dabb8b9ee46 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:55.326235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f788b6b-843e-496f-affa-4dabb8b9ee46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:55.327859 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f788b6b-843e-496f-affa-4dabb8b9ee46 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:55.337654 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1404/2851] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 21:12:55.342307 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e7c38-ed28-4a7b-a285-4b1c2df0952f admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:55.352135 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e7c38-ed28-4a7b-a285-4b1c2df0952f admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:55.357272 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede66341-d063-4c6d-ad04-e3fb60faac26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:55.358074 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede66341-d063-4c6d-ad04-e3fb60faac26 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:55.358352 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede66341-d063-4c6d-ad04-e3fb60faac26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:55.358700 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede66341-d063-4c6d-ad04-e3fb60faac26 admin admin] PATH_INFO: `/v3/projects/ec237851d9834938bdd003d05e0b13e4/users/2ad3cb9d9cf545fa80ed2ff031b38d1d/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:55.373986 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede66341-d063-4c6d-ad04-e3fb60faac26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec237851d9834938bdd003d05e0b13e4, user_id=2ad3cb9d9cf545fa80ed2ff031b38d1d, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:55.376629 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede66341-d063-4c6d-ad04-e3fb60faac26 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:55.389314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1448/2852] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:12:55 2022] PUT /identity/v3/projects/ec237851d9834938bdd003d05e0b13e4/users/2ad3cb9d9cf545fa80ed2ff031b38d1d/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:12:55.393343 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f788b6b-843e-496f-affa-4dabb8b9ee46 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:55.403968 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f788b6b-843e-496f-affa-4dabb8b9ee46 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:55.460124 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a84f-f73b-42ef-a8bb-9380554a305c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:55.460801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a84f-f73b-42ef-a8bb-9380554a305c admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:55.461052 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a84f-f73b-42ef-a8bb-9380554a305c admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:55.461326 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a84f-f73b-42ef-a8bb-9380554a305c admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:55.462053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37a84f-f73b-42ef-a8bb-9380554a305c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:55.463636 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37a84f-f73b-42ef-a8bb-9380554a305c admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:55.475297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1405/2853] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:12:55.479621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede66341-d063-4c6d-ad04-e3fb60faac26 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:55.490465 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede66341-d063-4c6d-ad04-e3fb60faac26 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:55.495098 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd5298-bdf2-4b70-9c74-2920ed81b95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:55.496262 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd5298-bdf2-4b70-9c74-2920ed81b95b admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:55.496546 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd5298-bdf2-4b70-9c74-2920ed81b95b admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:55.496825 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd5298-bdf2-4b70-9c74-2920ed81b95b admin admin] PATH_INFO: `/v3/projects/ec237851d9834938bdd003d05e0b13e4/users/2ad3cb9d9cf545fa80ed2ff031b38d1d/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:55.499605 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd5298-bdf2-4b70-9c74-2920ed81b95b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec237851d9834938bdd003d05e0b13e4, user_id=2ad3cb9d9cf545fa80ed2ff031b38d1d, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:55.502251 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd5298-bdf2-4b70-9c74-2920ed81b95b admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:55.512483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1449/2854] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:12:55 2022] PUT /identity/v3/projects/ec237851d9834938bdd003d05e0b13e4/users/2ad3cb9d9cf545fa80ed2ff031b38d1d/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 21:12:55.573336 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd875cd3-8483-4695-8dec-705c77721510 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:55.573649 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd875cd3-8483-4695-8dec-705c77721510 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:55.573932 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd875cd3-8483-4695-8dec-705c77721510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:55.601053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-dd875cd3-8483-4695-8dec-705c77721510 None None] MFA Rules not processed for user `2ad3cb9d9cf545fa80ed2ff031b38d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:12:55.602286 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-dd875cd3-8483-4695-8dec-705c77721510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:55.670632 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1406/2855] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:12:55 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:12:55.681073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd5298-bdf2-4b70-9c74-2920ed81b95b admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:55.682775 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-82bd5298-bdf2-4b70-9c74-2920ed81b95b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:55.739699 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd5298-bdf2-4b70-9c74-2920ed81b95b admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:55.743377 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25b19e-1842-4cb3-af17-dce9dbb1ffa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:55.744046 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25b19e-1842-4cb3-af17-dce9dbb1ffa4 service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:55.744338 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25b19e-1842-4cb3-af17-dce9dbb1ffa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:55.744615 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25b19e-1842-4cb3-af17-dce9dbb1ffa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:55.754592 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25b19e-1842-4cb3-af17-dce9dbb1ffa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:55.756055 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25b19e-1842-4cb3-af17-dce9dbb1ffa4 service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:55.766030 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1450/2856] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:12:55 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:12:55.908267 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd875cd3-8483-4695-8dec-705c77721510 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:55.969673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd875cd3-8483-4695-8dec-705c77721510 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:55.974160 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529c88c-de9f-416e-ab6e-4a6f66d3edd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:55.975017 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529c88c-de9f-416e-ab6e-4a6f66d3edd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:55.975292 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529c88c-de9f-416e-ab6e-4a6f66d3edd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:55.975563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529c88c-de9f-416e-ab6e-4a6f66d3edd2 admin admin] PATH_INFO: `/v3/users/3cb0ebc0a6c24599a32c8affcc7e4c42` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:55.976767 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529c88c-de9f-416e-ab6e-4a6f66d3edd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cb0ebc0a6c24599a32c8affcc7e4c42)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:55.978695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529c88c-de9f-416e-ab6e-4a6f66d3edd2 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:55.984573 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=100 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:56.025219 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6529c88c-de9f-416e-ab6e-4a6f66d3edd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cb0ebc0a6c24599a32c8affcc7e4c42'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:56.032286 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6529c88c-de9f-416e-ab6e-4a6f66d3edd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cb0ebc0a6c24599a32c8affcc7e4c42'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:56.032885 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6529c88c-de9f-416e-ab6e-4a6f66d3edd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cb0ebc0a6c24599a32c8affcc7e4c42'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:56.039562 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6529c88c-de9f-416e-ab6e-4a6f66d3edd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cb0ebc0a6c24599a32c8affcc7e4c42'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:56.040247 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-6529c88c-de9f-416e-ab6e-4a6f66d3edd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cb0ebc0a6c24599a32c8affcc7e4c42'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:56.057622 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1407/2857] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:12:55 2022] DELETE /identity/v3/users/3cb0ebc0a6c24599a32c8affcc7e4c42 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:12:56.062618 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab50732-080c-4e8f-b89f-5b6881cd4f52 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:56.063133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab50732-080c-4e8f-b89f-5b6881cd4f52 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:56.063246 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab50732-080c-4e8f-b89f-5b6881cd4f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:56.080511 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-cab50732-080c-4e8f-b89f-5b6881cd4f52 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:12:56.081733 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-cab50732-080c-4e8f-b89f-5b6881cd4f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:56.140120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1451/2858] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:12:56.149906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529c88c-de9f-416e-ab6e-4a6f66d3edd2 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:56.151280 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6529c88c-de9f-416e-ab6e-4a6f66d3edd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:56.209613 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529c88c-de9f-416e-ab6e-4a6f66d3edd2 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:56.213262 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3529c5-6873-4cba-8512-5b8fee098dda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:56.213947 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3529c5-6873-4cba-8512-5b8fee098dda service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:56.214461 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3529c5-6873-4cba-8512-5b8fee098dda service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:56.215041 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3529c5-6873-4cba-8512-5b8fee098dda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:56.226133 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3529c5-6873-4cba-8512-5b8fee098dda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:56.227491 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3529c5-6873-4cba-8512-5b8fee098dda service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:56.237588 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1408/2859] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:12:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:12:56.511591 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab50732-080c-4e8f-b89f-5b6881cd4f52 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:56.513826 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-cab50732-080c-4e8f-b89f-5b6881cd4f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:56.525791 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab50732-080c-4e8f-b89f-5b6881cd4f52 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:56.531133 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:56.532036 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:56.532317 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:56.532621 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2 admin admin] PATH_INFO: `/v3/projects/f1f44f5641154e3a82ed5e9c1c31bf83` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:56.533815 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1f44f5641154e3a82ed5e9c1c31bf83)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:56.535899 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:56.573971 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:56.585061 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1f44f5641154e3a82ed5e9c1c31bf83', 'request_id': 'req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:56.591814 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1f44f5641154e3a82ed5e9c1c31bf83', 'request_id': 'req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:56.597442 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1f44f5641154e3a82ed5e9c1c31bf83', 'request_id': 'req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:56.609741 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2 admin admin] The token cache is being invalidate because project f1f44f5641154e3a82ed5e9c1c31bf83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:12:56.610032 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:56.610661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:56.612233 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1452/2860] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:12:56 2022] DELETE /identity/v3/projects/f1f44f5641154e3a82ed5e9c1c31bf83 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:12:56.616304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3529c5-6873-4cba-8512-5b8fee098dda service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:56.626230 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3529c5-6873-4cba-8512-5b8fee098dda service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:56.684746 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d41f56-25e8-4b5c-988f-c8c3a08da568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:56.685633 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d41f56-25e8-4b5c-988f-c8c3a08da568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:56.685921 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d41f56-25e8-4b5c-988f-c8c3a08da568 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:56.686196 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d41f56-25e8-4b5c-988f-c8c3a08da568 admin admin] PATH_INFO: `/v3/users/2ad3cb9d9cf545fa80ed2ff031b38d1d` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:56.687408 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d41f56-25e8-4b5c-988f-c8c3a08da568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ad3cb9d9cf545fa80ed2ff031b38d1d)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:56.689358 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d41f56-25e8-4b5c-988f-c8c3a08da568 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:56.723038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:56.738095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-15d41f56-25e8-4b5c-988f-c8c3a08da568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ad3cb9d9cf545fa80ed2ff031b38d1d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:56.744949 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-15d41f56-25e8-4b5c-988f-c8c3a08da568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ad3cb9d9cf545fa80ed2ff031b38d1d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:56.745910 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-15d41f56-25e8-4b5c-988f-c8c3a08da568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ad3cb9d9cf545fa80ed2ff031b38d1d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:56.752946 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-15d41f56-25e8-4b5c-988f-c8c3a08da568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ad3cb9d9cf545fa80ed2ff031b38d1d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:56.753577 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-15d41f56-25e8-4b5c-988f-c8c3a08da568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ad3cb9d9cf545fa80ed2ff031b38d1d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:56.771315 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1409/2861] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:12:56 2022] DELETE /identity/v3/users/2ad3cb9d9cf545fa80ed2ff031b38d1d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:12:57.050185 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:57.051616 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:57.064788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf746e-6585-4f3a-b0a0-4ee02e1ce9a2 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:57.122205 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d3bd1-3b4c-448a-841e-2812dea3b5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:57.123065 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d3bd1-3b4c-448a-841e-2812dea3b5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:57.123351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d3bd1-3b4c-448a-841e-2812dea3b5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:57.123630 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d3bd1-3b4c-448a-841e-2812dea3b5b3 admin admin] PATH_INFO: `/v3/projects/ec237851d9834938bdd003d05e0b13e4` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:57.124843 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d3bd1-3b4c-448a-841e-2812dea3b5b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec237851d9834938bdd003d05e0b13e4)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:57.126781 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d3bd1-3b4c-448a-841e-2812dea3b5b3 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:57.172860 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c81d3bd1-3b4c-448a-841e-2812dea3b5b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec237851d9834938bdd003d05e0b13e4', 'request_id': 'req-c81d3bd1-3b4c-448a-841e-2812dea3b5b3'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:57.178822 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c81d3bd1-3b4c-448a-841e-2812dea3b5b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec237851d9834938bdd003d05e0b13e4', 'request_id': 'req-c81d3bd1-3b4c-448a-841e-2812dea3b5b3'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:57.184212 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c81d3bd1-3b4c-448a-841e-2812dea3b5b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec237851d9834938bdd003d05e0b13e4', 'request_id': 'req-c81d3bd1-3b4c-448a-841e-2812dea3b5b3'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:57.196231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c81d3bd1-3b4c-448a-841e-2812dea3b5b3 admin admin] The token cache is being invalidate because project ec237851d9834938bdd003d05e0b13e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:12:57.196515 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c81d3bd1-3b4c-448a-841e-2812dea3b5b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:57.197128 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-c81d3bd1-3b4c-448a-841e-2812dea3b5b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:12:57.198693 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1453/2862] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:12:57 2022] DELETE /identity/v3/projects/ec237851d9834938bdd003d05e0b13e4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:12:57.203741 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe502ee-923e-4523-9e7f-48afef511e8c None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:57.204248 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe502ee-923e-4523-9e7f-48afef511e8c None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:57.204382 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe502ee-923e-4523-9e7f-48afef511e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:57.223178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-6fe502ee-923e-4523-9e7f-48afef511e8c None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:12:57.224432 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6fe502ee-923e-4523-9e7f-48afef511e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:57.285238 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1410/2863] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:12:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:12:57.293064 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe825d34-ea6b-4f3d-8869-0869e57599e9 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:57.293292 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe825d34-ea6b-4f3d-8869-0869e57599e9 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:57.293602 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe825d34-ea6b-4f3d-8869-0869e57599e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:57.312892 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-fe825d34-ea6b-4f3d-8869-0869e57599e9 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:12:57.314094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-fe825d34-ea6b-4f3d-8869-0869e57599e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:57.321938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1454/2864] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:12:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:12:57.326162 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe502ee-923e-4523-9e7f-48afef511e8c None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:57.335936 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe502ee-923e-4523-9e7f-48afef511e8c None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:57.340513 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a6548-15f7-49cb-bbd8-ba941aaf2314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:57.341200 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a6548-15f7-49cb-bbd8-ba941aaf2314 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:57.341484 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a6548-15f7-49cb-bbd8-ba941aaf2314 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:57.341760 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a6548-15f7-49cb-bbd8-ba941aaf2314 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:57.342483 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a6548-15f7-49cb-bbd8-ba941aaf2314 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:12:57.342790 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a6548-15f7-49cb-bbd8-ba941aaf2314 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:57.344406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a6548-15f7-49cb-bbd8-ba941aaf2314 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:57.355936 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1411/2865] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:12:57.360573 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe825d34-ea6b-4f3d-8869-0869e57599e9 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:57.369639 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe825d34-ea6b-4f3d-8869-0869e57599e9 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:57.376116 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6eca9-922d-498e-990e-cdec224bceea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:57.376765 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6eca9-922d-498e-990e-cdec224bceea admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:57.377056 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6eca9-922d-498e-990e-cdec224bceea admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:57.377335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6eca9-922d-498e-990e-cdec224bceea admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:57.378038 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6eca9-922d-498e-990e-cdec224bceea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:57.379964 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6eca9-922d-498e-990e-cdec224bceea admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:57.406847 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1455/2866] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:12:57 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:12:57.410998 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a6548-15f7-49cb-bbd8-ba941aaf2314 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:57.421762 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a6548-15f7-49cb-bbd8-ba941aaf2314 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:57.479451 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316913f-f24f-4c1a-b40e-75e2147d3a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:57.480154 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316913f-f24f-4c1a-b40e-75e2147d3a4b admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:57.480447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316913f-f24f-4c1a-b40e-75e2147d3a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:57.480724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316913f-f24f-4c1a-b40e-75e2147d3a4b admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:57.481977 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316913f-f24f-4c1a-b40e-75e2147d3a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:57.484106 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316913f-f24f-4c1a-b40e-75e2147d3a4b admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:57.506089 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1412/2867] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:12:57 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 21:12:57.510194 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6eca9-922d-498e-990e-cdec224bceea admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:57.520996 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6eca9-922d-498e-990e-cdec224bceea admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:57.525610 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c429aaa-07fb-4c6b-a069-f5e0b72a1be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:57.526269 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c429aaa-07fb-4c6b-a069-f5e0b72a1be6 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:57.526572 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c429aaa-07fb-4c6b-a069-f5e0b72a1be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:57.526825 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c429aaa-07fb-4c6b-a069-f5e0b72a1be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:57.527550 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c429aaa-07fb-4c6b-a069-f5e0b72a1be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:57.529650 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c429aaa-07fb-4c6b-a069-f5e0b72a1be6 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:57.540434 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1456/2868] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:12:57.545188 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316913f-f24f-4c1a-b40e-75e2147d3a4b admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:57.555664 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316913f-f24f-4c1a-b40e-75e2147d3a4b admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:57.561418 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff243884-e0f7-4800-9128-73b38035a0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:57.562176 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff243884-e0f7-4800-9128-73b38035a0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:57.562487 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff243884-e0f7-4800-9128-73b38035a0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:57.562789 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff243884-e0f7-4800-9128-73b38035a0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:57.563550 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff243884-e0f7-4800-9128-73b38035a0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:57.565183 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff243884-e0f7-4800-9128-73b38035a0c1 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:57.578970 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1413/2869] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:12:57.580223 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c429aaa-07fb-4c6b-a069-f5e0b72a1be6 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:57.590351 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c429aaa-07fb-4c6b-a069-f5e0b72a1be6 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:57.595603 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dabdf-c5a0-4b7a-b3e0-e11679d1fb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:57.596514 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dabdf-c5a0-4b7a-b3e0-e11679d1fb05 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:57.596797 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dabdf-c5a0-4b7a-b3e0-e11679d1fb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:57.597076 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dabdf-c5a0-4b7a-b3e0-e11679d1fb05 admin admin] PATH_INFO: `/v3/projects/e6fae9d95032440fb07fe649fb3c3779/users/2aa34dd76d9e49b8bce277b9f4eff227/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:57.609682 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:57.613142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17dabdf-c5a0-4b7a-b3e0-e11679d1fb05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6fae9d95032440fb07fe649fb3c3779, user_id=2aa34dd76d9e49b8bce277b9f4eff227, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:57.615879 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17dabdf-c5a0-4b7a-b3e0-e11679d1fb05 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:57.630264 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1457/2870] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:12:57 2022] PUT /identity/v3/projects/e6fae9d95032440fb07fe649fb3c3779/users/2aa34dd76d9e49b8bce277b9f4eff227/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:12:57.640569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c9dc0-1aee-4872-acdf-d7d7725aa41d None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:57.640877 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c9dc0-1aee-4872-acdf-d7d7725aa41d None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:57.641149 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c9dc0-1aee-4872-acdf-d7d7725aa41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:57.666068 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:57.670248 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-ec5c9dc0-1aee-4872-acdf-d7d7725aa41d None None] MFA Rules not processed for user `2aa34dd76d9e49b8bce277b9f4eff227`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:12:57.672187 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-ec5c9dc0-1aee-4872-acdf-d7d7725aa41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:57.741423 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1414/2871] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:12:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:12:57.746142 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dabdf-c5a0-4b7a-b3e0-e11679d1fb05 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:57.809953 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dabdf-c5a0-4b7a-b3e0-e11679d1fb05 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:57.814300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e05a48-8d5f-4757-8f3f-1bfdec88db74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:57.814944 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e05a48-8d5f-4757-8f3f-1bfdec88db74 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:57.815224 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e05a48-8d5f-4757-8f3f-1bfdec88db74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:57.815670 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e05a48-8d5f-4757-8f3f-1bfdec88db74 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:57.816339 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e05a48-8d5f-4757-8f3f-1bfdec88db74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:57.818208 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e05a48-8d5f-4757-8f3f-1bfdec88db74 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:57.843203 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1458/2872] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:12:57 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:12:57.847409 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c9dc0-1aee-4872-acdf-d7d7725aa41d None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:57.858828 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c9dc0-1aee-4872-acdf-d7d7725aa41d None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:57.914104 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a2248-3bc8-4180-9008-4911491e3576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:57.914765 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a2248-3bc8-4180-9008-4911491e3576 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:57.915046 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a2248-3bc8-4180-9008-4911491e3576 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:57.915317 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a2248-3bc8-4180-9008-4911491e3576 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:57.916038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a2248-3bc8-4180-9008-4911491e3576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:57.917934 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a2248-3bc8-4180-9008-4911491e3576 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:57.938906 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1415/2873] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:12:57 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:12:57.942992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e05a48-8d5f-4757-8f3f-1bfdec88db74 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:57.953378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e05a48-8d5f-4757-8f3f-1bfdec88db74 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:57.958147 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5f518-3102-41bb-b59c-a1d9e51cd41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:57.958804 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5f518-3102-41bb-b59c-a1d9e51cd41e admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:57.959088 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5f518-3102-41bb-b59c-a1d9e51cd41e admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:57.959365 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5f518-3102-41bb-b59c-a1d9e51cd41e admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:57.960122 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5f518-3102-41bb-b59c-a1d9e51cd41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:57.961757 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5f518-3102-41bb-b59c-a1d9e51cd41e admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:57.972665 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1459/2874] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:12:57.976726 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a2248-3bc8-4180-9008-4911491e3576 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:57.987297 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a2248-3bc8-4180-9008-4911491e3576 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:57.992014 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e132701-b30e-48e7-ad40-18bdeff83dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:57.992893 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e132701-b30e-48e7-ad40-18bdeff83dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:57.993177 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e132701-b30e-48e7-ad40-18bdeff83dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:57.993456 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e132701-b30e-48e7-ad40-18bdeff83dbc admin admin] PATH_INFO: `/v3/projects/f1b397c43d7745c3960b7b5ec1bceb6b/users/f08e678063aa44f38641ec4b3423b7f3/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:58.008852 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e132701-b30e-48e7-ad40-18bdeff83dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b397c43d7745c3960b7b5ec1bceb6b, user_id=f08e678063aa44f38641ec4b3423b7f3, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:58.011478 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e132701-b30e-48e7-ad40-18bdeff83dbc admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:58.024722 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1416/2875] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:12:57 2022] PUT /identity/v3/projects/f1b397c43d7745c3960b7b5ec1bceb6b/users/f08e678063aa44f38641ec4b3423b7f3/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:12:58.029435 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5f518-3102-41bb-b59c-a1d9e51cd41e admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:58.039336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5f518-3102-41bb-b59c-a1d9e51cd41e admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:58.093596 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c1446-1a78-4deb-b3af-f52835980566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:58.094243 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c1446-1a78-4deb-b3af-f52835980566 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:58.094516 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c1446-1a78-4deb-b3af-f52835980566 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:58.094786 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c1446-1a78-4deb-b3af-f52835980566 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:58.095505 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c1446-1a78-4deb-b3af-f52835980566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:58.097114 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c1446-1a78-4deb-b3af-f52835980566 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:58.106859 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1460/2876] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:12:58.111058 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e132701-b30e-48e7-ad40-18bdeff83dbc admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:58.121477 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e132701-b30e-48e7-ad40-18bdeff83dbc admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:58.126344 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339329a-34f0-42cd-b7e6-ae8017b68912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:58.127212 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339329a-34f0-42cd-b7e6-ae8017b68912 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:58.127503 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339329a-34f0-42cd-b7e6-ae8017b68912 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:58.127812 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339329a-34f0-42cd-b7e6-ae8017b68912 admin admin] PATH_INFO: `/v3/projects/f1b397c43d7745c3960b7b5ec1bceb6b/users/f08e678063aa44f38641ec4b3423b7f3/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:58.129793 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339329a-34f0-42cd-b7e6-ae8017b68912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b397c43d7745c3960b7b5ec1bceb6b, user_id=f08e678063aa44f38641ec4b3423b7f3, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:58.132456 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339329a-34f0-42cd-b7e6-ae8017b68912 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:58.143095 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1417/2877] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:12:58 2022] PUT /identity/v3/projects/f1b397c43d7745c3960b7b5ec1bceb6b/users/f08e678063aa44f38641ec4b3423b7f3/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:12:58.154970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f67e8d-8738-47b8-b561-848768b6ccae None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:58.155258 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f67e8d-8738-47b8-b561-848768b6ccae None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:58.155534 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f67e8d-8738-47b8-b561-848768b6ccae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:58.181492 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-a2f67e8d-8738-47b8-b561-848768b6ccae None None] MFA Rules not processed for user `f08e678063aa44f38641ec4b3423b7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:12:58.182722 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a2f67e8d-8738-47b8-b561-848768b6ccae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:58.228910 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:58.250593 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1461/2878] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:12:58 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:12:58.262257 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339329a-34f0-42cd-b7e6-ae8017b68912 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:58.264582 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-e339329a-34f0-42cd-b7e6-ae8017b68912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:12:58.321074 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339329a-34f0-42cd-b7e6-ae8017b68912 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:58.324766 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0966017-9627-4376-a1d3-72bf65a33219 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:58.325404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0966017-9627-4376-a1d3-72bf65a33219 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:58.325690 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0966017-9627-4376-a1d3-72bf65a33219 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:58.326462 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0966017-9627-4376-a1d3-72bf65a33219 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:58.335626 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0966017-9627-4376-a1d3-72bf65a33219 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:58.337087 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0966017-9627-4376-a1d3-72bf65a33219 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:58.341040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:12:58.349791 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1418/2879] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:12:58 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:12:58.423620 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f67e8d-8738-47b8-b561-848768b6ccae None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:12:58.433679 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f67e8d-8738-47b8-b561-848768b6ccae None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:12:58.438131 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52f825-10d4-4e0b-a496-4710669482fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:12:58.438774 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52f825-10d4-4e0b-a496-4710669482fe service cinder] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:12:58.439056 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52f825-10d4-4e0b-a496-4710669482fe service cinder] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:12:58.439329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52f825-10d4-4e0b-a496-4710669482fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:12:58.496378 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52f825-10d4-4e0b-a496-4710669482fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:12:58.497788 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52f825-10d4-4e0b-a496-4710669482fe service cinder] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:12:58.507223 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1462/2880] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:12:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:13:08.350215 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:13:08.503214 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=34 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:13:43.075098 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0966017-9627-4376-a1d3-72bf65a33219 service cinder] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:43.137404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0966017-9627-4376-a1d3-72bf65a33219 service cinder] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:43.141883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730a1de-ee44-408c-ba94-8162e23a3c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:43.142731 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730a1de-ee44-408c-ba94-8162e23a3c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:43.143010 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730a1de-ee44-408c-ba94-8162e23a3c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:43.143282 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730a1de-ee44-408c-ba94-8162e23a3c11 admin admin] PATH_INFO: `/v3/users/2aa34dd76d9e49b8bce277b9f4eff227` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:43.145071 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730a1de-ee44-408c-ba94-8162e23a3c11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aa34dd76d9e49b8bce277b9f4eff227)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:43.147040 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730a1de-ee44-408c-ba94-8162e23a3c11 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:43.191996 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-f730a1de-ee44-408c-ba94-8162e23a3c11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aa34dd76d9e49b8bce277b9f4eff227'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:43.199410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-f730a1de-ee44-408c-ba94-8162e23a3c11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aa34dd76d9e49b8bce277b9f4eff227'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:43.200695 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-f730a1de-ee44-408c-ba94-8162e23a3c11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aa34dd76d9e49b8bce277b9f4eff227'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:43.208724 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-f730a1de-ee44-408c-ba94-8162e23a3c11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aa34dd76d9e49b8bce277b9f4eff227'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:43.209365 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-f730a1de-ee44-408c-ba94-8162e23a3c11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aa34dd76d9e49b8bce277b9f4eff227'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:43.226528 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1419/2881] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:13:43 2022] DELETE /identity/v3/users/2aa34dd76d9e49b8bce277b9f4eff227 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:13:43.230970 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96565741-c4c1-43da-b302-3a0b56008220 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:43.231482 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96565741-c4c1-43da-b302-3a0b56008220 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:43.231633 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96565741-c4c1-43da-b302-3a0b56008220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:43.248471 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-96565741-c4c1-43da-b302-3a0b56008220 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:13:43.249667 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-96565741-c4c1-43da-b302-3a0b56008220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:43.309473 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1463/2882] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:13:43.320135 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730a1de-ee44-408c-ba94-8162e23a3c11 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:43.321557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-f730a1de-ee44-408c-ba94-8162e23a3c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:43.385554 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730a1de-ee44-408c-ba94-8162e23a3c11 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:43.389229 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81805b8-cb2f-467e-8426-c5a6229c92b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:43.389871 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81805b8-cb2f-467e-8426-c5a6229c92b5 service neutron] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:43.390156 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81805b8-cb2f-467e-8426-c5a6229c92b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:43.390432 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81805b8-cb2f-467e-8426-c5a6229c92b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:43.400263 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81805b8-cb2f-467e-8426-c5a6229c92b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:43.401697 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81805b8-cb2f-467e-8426-c5a6229c92b5 service neutron] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:43.412485 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1420/2883] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:13:43.678393 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96565741-c4c1-43da-b302-3a0b56008220 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:43.679938 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-96565741-c4c1-43da-b302-3a0b56008220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:43.691940 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96565741-c4c1-43da-b302-3a0b56008220 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:43.696802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e76ef0-df33-4e01-bb11-867b082bc272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:43.697659 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e76ef0-df33-4e01-bb11-867b082bc272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:43.697943 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e76ef0-df33-4e01-bb11-867b082bc272 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:43.698220 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e76ef0-df33-4e01-bb11-867b082bc272 admin admin] PATH_INFO: `/v3/projects/e6fae9d95032440fb07fe649fb3c3779` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:43.699417 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e76ef0-df33-4e01-bb11-867b082bc272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6fae9d95032440fb07fe649fb3c3779)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:43.701414 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e76ef0-df33-4e01-bb11-867b082bc272 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:43.756332 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-a3e76ef0-df33-4e01-bb11-867b082bc272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6fae9d95032440fb07fe649fb3c3779', 'request_id': 'req-a3e76ef0-df33-4e01-bb11-867b082bc272'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:43.762507 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-a3e76ef0-df33-4e01-bb11-867b082bc272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6fae9d95032440fb07fe649fb3c3779', 'request_id': 'req-a3e76ef0-df33-4e01-bb11-867b082bc272'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:43.768070 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-a3e76ef0-df33-4e01-bb11-867b082bc272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6fae9d95032440fb07fe649fb3c3779', 'request_id': 'req-a3e76ef0-df33-4e01-bb11-867b082bc272'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:43.780523 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-a3e76ef0-df33-4e01-bb11-867b082bc272 admin admin] The token cache is being invalidate because project e6fae9d95032440fb07fe649fb3c3779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:13:43.780818 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-a3e76ef0-df33-4e01-bb11-867b082bc272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:43.781445 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-a3e76ef0-df33-4e01-bb11-867b082bc272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:43.783096 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1464/2884] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:13:43 2022] DELETE /identity/v3/projects/e6fae9d95032440fb07fe649fb3c3779 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:13:43.787404 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81805b8-cb2f-467e-8426-c5a6229c92b5 service neutron] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:43.797239 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81805b8-cb2f-467e-8426-c5a6229c92b5 service neutron] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:43.817086 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:13:43.855545 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455a579-db42-443c-851a-f76c1435db0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:43.857807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455a579-db42-443c-851a-f76c1435db0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:43.857807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455a579-db42-443c-851a-f76c1435db0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:43.857807 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455a579-db42-443c-851a-f76c1435db0f admin admin] PATH_INFO: `/v3/users/f08e678063aa44f38641ec4b3423b7f3` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:43.858373 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455a579-db42-443c-851a-f76c1435db0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f08e678063aa44f38641ec4b3423b7f3)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:43.860327 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455a579-db42-443c-851a-f76c1435db0f admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:43.906938 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b455a579-db42-443c-851a-f76c1435db0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f08e678063aa44f38641ec4b3423b7f3'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:43.914672 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b455a579-db42-443c-851a-f76c1435db0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f08e678063aa44f38641ec4b3423b7f3'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:43.915389 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b455a579-db42-443c-851a-f76c1435db0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f08e678063aa44f38641ec4b3423b7f3'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:43.922279 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b455a579-db42-443c-851a-f76c1435db0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f08e678063aa44f38641ec4b3423b7f3'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:43.922905 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-b455a579-db42-443c-851a-f76c1435db0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f08e678063aa44f38641ec4b3423b7f3'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:43.940130 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1421/2885] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:13:43 2022] DELETE /identity/v3/users/f08e678063aa44f38641ec4b3423b7f3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:13:44.215234 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e76ef0-df33-4e01-bb11-867b082bc272 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:44.216846 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a3e76ef0-df33-4e01-bb11-867b082bc272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:44.232845 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e76ef0-df33-4e01-bb11-867b082bc272 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:44.292336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83c0c6-698e-4b70-82c6-f370c6300ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:44.293117 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83c0c6-698e-4b70-82c6-f370c6300ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:44.293403 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83c0c6-698e-4b70-82c6-f370c6300ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:44.293683 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83c0c6-698e-4b70-82c6-f370c6300ec3 admin admin] PATH_INFO: `/v3/projects/f1b397c43d7745c3960b7b5ec1bceb6b` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:44.294869 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83c0c6-698e-4b70-82c6-f370c6300ec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b397c43d7745c3960b7b5ec1bceb6b)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:44.296809 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83c0c6-698e-4b70-82c6-f370c6300ec3 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:44.326561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:13:44.344146 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-df83c0c6-698e-4b70-82c6-f370c6300ec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b397c43d7745c3960b7b5ec1bceb6b', 'request_id': 'req-df83c0c6-698e-4b70-82c6-f370c6300ec3'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:44.350762 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-df83c0c6-698e-4b70-82c6-f370c6300ec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b397c43d7745c3960b7b5ec1bceb6b', 'request_id': 'req-df83c0c6-698e-4b70-82c6-f370c6300ec3'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:44.356329 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-df83c0c6-698e-4b70-82c6-f370c6300ec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b397c43d7745c3960b7b5ec1bceb6b', 'request_id': 'req-df83c0c6-698e-4b70-82c6-f370c6300ec3'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:44.369189 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-df83c0c6-698e-4b70-82c6-f370c6300ec3 admin admin] The token cache is being invalidate because project f1b397c43d7745c3960b7b5ec1bceb6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87061) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:13:44.369478 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-df83c0c6-698e-4b70-82c6-f370c6300ec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:44.370120 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-df83c0c6-698e-4b70-82c6-f370c6300ec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:44.372002 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1465/2886] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:13:44 2022] DELETE /identity/v3/projects/f1b397c43d7745c3960b7b5ec1bceb6b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:13:44.377333 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddd2cd-02a8-4c7c-b9d3-9e82bb7b3f5f None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:44.378053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddd2cd-02a8-4c7c-b9d3-9e82bb7b3f5f None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:44.378053 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddd2cd-02a8-4c7c-b9d3-9e82bb7b3f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:44.396291 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-05ddd2cd-02a8-4c7c-b9d3-9e82bb7b3f5f None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:13:44.397501 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-05ddd2cd-02a8-4c7c-b9d3-9e82bb7b3f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:44.458301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1422/2887] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:13:44.466716 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ab58d-af06-4611-912e-0a80620ae143 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:44.467006 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ab58d-af06-4611-912e-0a80620ae143 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:44.467289 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ab58d-af06-4611-912e-0a80620ae143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:44.485713 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-359ab58d-af06-4611-912e-0a80620ae143 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:13:44.486916 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-359ab58d-af06-4611-912e-0a80620ae143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:44.494737 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1466/2888] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:13:44.499122 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ddd2cd-02a8-4c7c-b9d3-9e82bb7b3f5f None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:44.509093 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ddd2cd-02a8-4c7c-b9d3-9e82bb7b3f5f None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:44.513613 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b97096-96cc-461a-9cc2-249182e3f7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:44.514260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b97096-96cc-461a-9cc2-249182e3f7a7 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:44.514545 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b97096-96cc-461a-9cc2-249182e3f7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:44.514820 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b97096-96cc-461a-9cc2-249182e3f7a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:44.515546 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b97096-96cc-461a-9cc2-249182e3f7a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:13:44.515887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b97096-96cc-461a-9cc2-249182e3f7a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:44.517551 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b97096-96cc-461a-9cc2-249182e3f7a7 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:44.530378 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1423/2889] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 21:13:44.534561 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ab58d-af06-4611-912e-0a80620ae143 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:44.544335 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ab58d-af06-4611-912e-0a80620ae143 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:44.548869 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8e32f-9df7-4c05-9739-a4111b1b9840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:44.549505 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8e32f-9df7-4c05-9739-a4111b1b9840 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:44.549787 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8e32f-9df7-4c05-9739-a4111b1b9840 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:44.550063 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8e32f-9df7-4c05-9739-a4111b1b9840 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:44.550744 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8e32f-9df7-4c05-9739-a4111b1b9840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:44.552678 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8e32f-9df7-4c05-9739-a4111b1b9840 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:44.586929 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1467/2890] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:13:44 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:13:44.591155 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b97096-96cc-461a-9cc2-249182e3f7a7 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:44.601771 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b97096-96cc-461a-9cc2-249182e3f7a7 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:44.628687 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:13:44.660569 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b22fee-7aac-4a30-94cf-b2f41605ba2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:44.661305 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b22fee-7aac-4a30-94cf-b2f41605ba2e admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:44.661596 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b22fee-7aac-4a30-94cf-b2f41605ba2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:44.661877 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b22fee-7aac-4a30-94cf-b2f41605ba2e admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:44.662581 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b22fee-7aac-4a30-94cf-b2f41605ba2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:44.664547 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b22fee-7aac-4a30-94cf-b2f41605ba2e admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:44.684929 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1424/2891] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:13:44 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:13:44.689094 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8e32f-9df7-4c05-9739-a4111b1b9840 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:44.699948 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8e32f-9df7-4c05-9739-a4111b1b9840 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:44.704582 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd32e0-dddc-4c17-b233-eb7aca307912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:44.705260 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd32e0-dddc-4c17-b233-eb7aca307912 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:44.705536 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd32e0-dddc-4c17-b233-eb7aca307912 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:44.705817 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd32e0-dddc-4c17-b233-eb7aca307912 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:44.706535 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd32e0-dddc-4c17-b233-eb7aca307912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:44.708403 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd32e0-dddc-4c17-b233-eb7aca307912 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:44.718253 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1468/2892] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:13:44.722091 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b22fee-7aac-4a30-94cf-b2f41605ba2e admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:44.732541 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b22fee-7aac-4a30-94cf-b2f41605ba2e admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:44.737381 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8dba9c-bf8b-4ce7-9246-b6b6cc5d605b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:44.738779 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8dba9c-bf8b-4ce7-9246-b6b6cc5d605b admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:44.739066 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8dba9c-bf8b-4ce7-9246-b6b6cc5d605b admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:44.739343 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8dba9c-bf8b-4ce7-9246-b6b6cc5d605b admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:44.740099 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8dba9c-bf8b-4ce7-9246-b6b6cc5d605b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:44.741673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8dba9c-bf8b-4ce7-9246-b6b6cc5d605b admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:44.751987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1425/2893] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:13:44.756008 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd32e0-dddc-4c17-b233-eb7aca307912 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:44.765888 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd32e0-dddc-4c17-b233-eb7aca307912 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:44.770802 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62023538-ec8f-4338-b4fc-f02de0939659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:44.771673 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62023538-ec8f-4338-b4fc-f02de0939659 admin admin] REQUEST_METHOD: `PUT` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:44.771992 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62023538-ec8f-4338-b4fc-f02de0939659 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:44.772273 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62023538-ec8f-4338-b4fc-f02de0939659 admin admin] PATH_INFO: `/v3/projects/e7d0ad8b117e4bc7a384c4ba30f0109c/users/268ddbdf85c646629b45a94d33af1903/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:44.788003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62023538-ec8f-4338-b4fc-f02de0939659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7d0ad8b117e4bc7a384c4ba30f0109c, user_id=268ddbdf85c646629b45a94d33af1903, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:44.790642 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62023538-ec8f-4338-b4fc-f02de0939659 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:44.803568 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1469/2894] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:13:44 2022] PUT /identity/v3/projects/e7d0ad8b117e4bc7a384c4ba30f0109c/users/268ddbdf85c646629b45a94d33af1903/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:13:44.813831 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d674d-6794-4398-8009-963135a527c4 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:44.814127 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d674d-6794-4398-8009-963135a527c4 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:44.814406 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d674d-6794-4398-8009-963135a527c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:44.840722 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-dd5d674d-6794-4398-8009-963135a527c4 None None] MFA Rules not processed for user `268ddbdf85c646629b45a94d33af1903`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:13:44.841927 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-dd5d674d-6794-4398-8009-963135a527c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:44.905438 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1426/2895] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:13:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:13:44.910483 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62023538-ec8f-4338-b4fc-f02de0939659 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:44.970823 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62023538-ec8f-4338-b4fc-f02de0939659 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:44.976294 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd969ed9-6215-4d40-8e40-48658f22a736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:44.976948 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd969ed9-6215-4d40-8e40-48658f22a736 admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:44.977231 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd969ed9-6215-4d40-8e40-48658f22a736 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:44.977509 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd969ed9-6215-4d40-8e40-48658f22a736 admin admin] PATH_INFO: `/v3/projects` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:44.978192 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd969ed9-6215-4d40-8e40-48658f22a736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:44.980102 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd969ed9-6215-4d40-8e40-48658f22a736 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:45.005731 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1470/2896] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:13:44 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:13:45.009985 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d674d-6794-4398-8009-963135a527c4 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:45.020887 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d674d-6794-4398-8009-963135a527c4 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:45.077496 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d094082-10c7-47ac-adf8-e6231409ead9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:45.078165 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d094082-10c7-47ac-adf8-e6231409ead9 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:45.078486 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d094082-10c7-47ac-adf8-e6231409ead9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:45.078763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d094082-10c7-47ac-adf8-e6231409ead9 admin admin] PATH_INFO: `/v3/users` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:45.079456 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d094082-10c7-47ac-adf8-e6231409ead9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:45.081384 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d094082-10c7-47ac-adf8-e6231409ead9 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:45.104295 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1427/2897] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:13:45 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 21:13:45.108885 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd969ed9-6215-4d40-8e40-48658f22a736 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:45.120037 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd969ed9-6215-4d40-8e40-48658f22a736 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:45.125124 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92512ff-79d0-403f-99c8-bad0d56c6b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:45.125789 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92512ff-79d0-403f-99c8-bad0d56c6b9c admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:45.126075 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92512ff-79d0-403f-99c8-bad0d56c6b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:45.126355 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92512ff-79d0-403f-99c8-bad0d56c6b9c admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:45.127108 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92512ff-79d0-403f-99c8-bad0d56c6b9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:45.128732 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92512ff-79d0-403f-99c8-bad0d56c6b9c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:45.139159 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1471/2898] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:13:45.143509 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d094082-10c7-47ac-adf8-e6231409ead9 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:45.154178 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d094082-10c7-47ac-adf8-e6231409ead9 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:45.158758 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f38f3-e3eb-48e5-a252-e28d8ec7f6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:45.159630 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f38f3-e3eb-48e5-a252-e28d8ec7f6ca admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:45.160057 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f38f3-e3eb-48e5-a252-e28d8ec7f6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:45.161044 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f38f3-e3eb-48e5-a252-e28d8ec7f6ca admin admin] PATH_INFO: `/v3/projects/9d4480c2d4ed4078b9f51264aa2272a9/users/f55a6ef53482475eb0c85e56a1be7144/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:45.175594 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f38f3-e3eb-48e5-a252-e28d8ec7f6ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d4480c2d4ed4078b9f51264aa2272a9, user_id=f55a6ef53482475eb0c85e56a1be7144, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:45.178253 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f38f3-e3eb-48e5-a252-e28d8ec7f6ca admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:45.191300 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1428/2899] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:13:45 2022] PUT /identity/v3/projects/9d4480c2d4ed4078b9f51264aa2272a9/users/f55a6ef53482475eb0c85e56a1be7144/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:13:45.195282 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92512ff-79d0-403f-99c8-bad0d56c6b9c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:45.205191 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92512ff-79d0-403f-99c8-bad0d56c6b9c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:45.217704 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:13:45.266186 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536144de-8432-446a-8df2-c6ffa212819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:45.266888 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536144de-8432-446a-8df2-c6ffa212819a admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:45.267134 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536144de-8432-446a-8df2-c6ffa212819a admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:45.267436 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536144de-8432-446a-8df2-c6ffa212819a admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:45.268211 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536144de-8432-446a-8df2-c6ffa212819a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:45.270500 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536144de-8432-446a-8df2-c6ffa212819a admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:45.281183 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1472/2900] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:13:45.286374 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f38f3-e3eb-48e5-a252-e28d8ec7f6ca admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:45.291916 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:13:45.298425 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f38f3-e3eb-48e5-a252-e28d8ec7f6ca admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:45.303612 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8a7d7-9efd-47c8-bca8-5add971d7a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:45.304525 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8a7d7-9efd-47c8-bca8-5add971d7a24 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:45.304812 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8a7d7-9efd-47c8-bca8-5add971d7a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:45.305092 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8a7d7-9efd-47c8-bca8-5add971d7a24 admin admin] PATH_INFO: `/v3/projects/9d4480c2d4ed4078b9f51264aa2272a9/users/f55a6ef53482475eb0c85e56a1be7144/roles/5e477269dd954c1d9bd08af262e355a1` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:45.307102 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8a7d7-9efd-47c8-bca8-5add971d7a24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d4480c2d4ed4078b9f51264aa2272a9, user_id=f55a6ef53482475eb0c85e56a1be7144, role_id=5e477269dd954c1d9bd08af262e355a1)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:45.309972 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8a7d7-9efd-47c8-bca8-5add971d7a24 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:45.320289 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1429/2901] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:13:45 2022] PUT /identity/v3/projects/9d4480c2d4ed4078b9f51264aa2272a9/users/f55a6ef53482475eb0c85e56a1be7144/roles/5e477269dd954c1d9bd08af262e355a1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:13:45.330388 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f0b7e-5264-480b-879c-3af8114742ae None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:45.330599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f0b7e-5264-480b-879c-3af8114742ae None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:45.330876 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f0b7e-5264-480b-879c-3af8114742ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:45.357885 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-ac8f0b7e-5264-480b-879c-3af8114742ae None None] MFA Rules not processed for user `f55a6ef53482475eb0c85e56a1be7144`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:13:45.359081 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ac8f0b7e-5264-480b-879c-3af8114742ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:45.434627 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1473/2902] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:13:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:13:45.446153 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8a7d7-9efd-47c8-bca8-5add971d7a24 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:45.447883 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-05f8a7d7-9efd-47c8-bca8-5add971d7a24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:45.505082 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8a7d7-9efd-47c8-bca8-5add971d7a24 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:45.508663 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f03d1d-bca0-4e34-b768-743ad7b05806 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8eef7746fc4417e8d4de0b052f3eaa0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:45.509303 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f03d1d-bca0-4e34-b768-743ad7b05806 service cinder] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:45.509584 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f03d1d-bca0-4e34-b768-743ad7b05806 service cinder] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:45.509854 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f03d1d-bca0-4e34-b768-743ad7b05806 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:45.518755 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f03d1d-bca0-4e34-b768-743ad7b05806 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:45.520987 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f03d1d-bca0-4e34-b768-743ad7b05806 service cinder] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:45.530283 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1430/2903] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:13:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:13:45.558061 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f0b7e-5264-480b-879c-3af8114742ae None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:45.621067 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f0b7e-5264-480b-879c-3af8114742ae None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:45.625666 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887c3ea-e98e-4e69-bf4e-b0bdaf763739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:45.626533 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887c3ea-e98e-4e69-bf4e-b0bdaf763739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:45.626819 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887c3ea-e98e-4e69-bf4e-b0bdaf763739 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:45.627099 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887c3ea-e98e-4e69-bf4e-b0bdaf763739 admin admin] PATH_INFO: `/v3/users/268ddbdf85c646629b45a94d33af1903` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:45.628334 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887c3ea-e98e-4e69-bf4e-b0bdaf763739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=268ddbdf85c646629b45a94d33af1903)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:45.630249 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887c3ea-e98e-4e69-bf4e-b0bdaf763739 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:45.671069 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:13:45.678441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9887c3ea-e98e-4e69-bf4e-b0bdaf763739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '268ddbdf85c646629b45a94d33af1903'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:45.679106 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9887c3ea-e98e-4e69-bf4e-b0bdaf763739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '268ddbdf85c646629b45a94d33af1903'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:45.690737 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9887c3ea-e98e-4e69-bf4e-b0bdaf763739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '268ddbdf85c646629b45a94d33af1903'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:45.698707 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9887c3ea-e98e-4e69-bf4e-b0bdaf763739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '268ddbdf85c646629b45a94d33af1903'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:45.705608 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-9887c3ea-e98e-4e69-bf4e-b0bdaf763739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '268ddbdf85c646629b45a94d33af1903'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:45.712347 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1474/2904] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:13:45 2022] DELETE /identity/v3/users/268ddbdf85c646629b45a94d33af1903 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:13:45.716899 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32b807-3c5d-42a9-bbc8-2844f6929ee5 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:45.717185 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32b807-3c5d-42a9-bbc8-2844f6929ee5 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:45.717460 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32b807-3c5d-42a9-bbc8-2844f6929ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:45.735047 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-6b32b807-3c5d-42a9-bbc8-2844f6929ee5 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:13:45.736260 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6b32b807-3c5d-42a9-bbc8-2844f6929ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:45.793231 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1431/2905] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 21:13:45.803531 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887c3ea-e98e-4e69-bf4e-b0bdaf763739 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:45.805031 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-9887c3ea-e98e-4e69-bf4e-b0bdaf763739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:45.861475 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887c3ea-e98e-4e69-bf4e-b0bdaf763739 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:45.865696 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ab1f1-8666-4eea-a357-b4a0c7b5adca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:45.866336 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ab1f1-8666-4eea-a357-b4a0c7b5adca service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:45.866620 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ab1f1-8666-4eea-a357-b4a0c7b5adca service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:45.866919 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ab1f1-8666-4eea-a357-b4a0c7b5adca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:45.877169 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ab1f1-8666-4eea-a357-b4a0c7b5adca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:45.878541 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ab1f1-8666-4eea-a357-b4a0c7b5adca service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:45.888655 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1475/2906] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:13:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 21:13:46.152067 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32b807-3c5d-42a9-bbc8-2844f6929ee5 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:46.153557 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6b32b807-3c5d-42a9-bbc8-2844f6929ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:46.167076 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32b807-3c5d-42a9-bbc8-2844f6929ee5 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:46.171980 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bd105-5261-4227-8cd9-b29723203c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:46.173004 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bd105-5261-4227-8cd9-b29723203c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:46.173288 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bd105-5261-4227-8cd9-b29723203c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:46.173563 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bd105-5261-4227-8cd9-b29723203c8d admin admin] PATH_INFO: `/v3/projects/e7d0ad8b117e4bc7a384c4ba30f0109c` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:46.174778 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971bd105-5261-4227-8cd9-b29723203c8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7d0ad8b117e4bc7a384c4ba30f0109c)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:46.177594 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971bd105-5261-4227-8cd9-b29723203c8d admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:46.225175 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-971bd105-5261-4227-8cd9-b29723203c8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7d0ad8b117e4bc7a384c4ba30f0109c', 'request_id': 'req-971bd105-5261-4227-8cd9-b29723203c8d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:46.231691 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-971bd105-5261-4227-8cd9-b29723203c8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7d0ad8b117e4bc7a384c4ba30f0109c', 'request_id': 'req-971bd105-5261-4227-8cd9-b29723203c8d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:46.237271 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-971bd105-5261-4227-8cd9-b29723203c8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7d0ad8b117e4bc7a384c4ba30f0109c', 'request_id': 'req-971bd105-5261-4227-8cd9-b29723203c8d'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:46.249163 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-971bd105-5261-4227-8cd9-b29723203c8d admin admin] The token cache is being invalidate because project e7d0ad8b117e4bc7a384c4ba30f0109c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:13:46.249449 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-971bd105-5261-4227-8cd9-b29723203c8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:46.250068 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-971bd105-5261-4227-8cd9-b29723203c8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:46.251627 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1432/2907] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:13:46 2022] DELETE /identity/v3/projects/e7d0ad8b117e4bc7a384c4ba30f0109c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:13:46.256534 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ab1f1-8666-4eea-a357-b4a0c7b5adca service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:46.266314 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ab1f1-8666-4eea-a357-b4a0c7b5adca service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:46.322908 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8c0d6-b4c5-499e-a1d9-1d60db54811d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:46.323823 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8c0d6-b4c5-499e-a1d9-1d60db54811d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:46.324107 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8c0d6-b4c5-499e-a1d9-1d60db54811d admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:46.324416 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8c0d6-b4c5-499e-a1d9-1d60db54811d admin admin] PATH_INFO: `/v3/users/f55a6ef53482475eb0c85e56a1be7144` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:46.325632 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8c0d6-b4c5-499e-a1d9-1d60db54811d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f55a6ef53482475eb0c85e56a1be7144)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:46.327599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8c0d6-b4c5-499e-a1d9-1d60db54811d admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:46.373911 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-34b8c0d6-b4c5-499e-a1d9-1d60db54811d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f55a6ef53482475eb0c85e56a1be7144'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:46.374579 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-34b8c0d6-b4c5-499e-a1d9-1d60db54811d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f55a6ef53482475eb0c85e56a1be7144'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:46.385104 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:13:46.387490 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-34b8c0d6-b4c5-499e-a1d9-1d60db54811d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f55a6ef53482475eb0c85e56a1be7144'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:46.395782 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-34b8c0d6-b4c5-499e-a1d9-1d60db54811d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f55a6ef53482475eb0c85e56a1be7144'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:46.402956 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-34b8c0d6-b4c5-499e-a1d9-1d60db54811d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f55a6ef53482475eb0c85e56a1be7144'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:46.409021 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1476/2908] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:13:46 2022] DELETE /identity/v3/users/f55a6ef53482475eb0c85e56a1be7144 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:13:46.683650 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bd105-5261-4227-8cd9-b29723203c8d admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:46.685210 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-971bd105-5261-4227-8cd9-b29723203c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:46.700941 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bd105-5261-4227-8cd9-b29723203c8d admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:46.714366 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:13:46.763579 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99cbc5-bd95-4f8e-b3c2-ef3efe307208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:46.766107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99cbc5-bd95-4f8e-b3c2-ef3efe307208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:46.766107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99cbc5-bd95-4f8e-b3c2-ef3efe307208 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:46.766107 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99cbc5-bd95-4f8e-b3c2-ef3efe307208 admin admin] PATH_INFO: `/v3/projects/9d4480c2d4ed4078b9f51264aa2272a9` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:46.766469 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d99cbc5-bd95-4f8e-b3c2-ef3efe307208 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d4480c2d4ed4078b9f51264aa2272a9)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:46.768301 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d99cbc5-bd95-4f8e-b3c2-ef3efe307208 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:46.814601 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9d99cbc5-bd95-4f8e-b3c2-ef3efe307208 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d4480c2d4ed4078b9f51264aa2272a9', 'request_id': 'req-9d99cbc5-bd95-4f8e-b3c2-ef3efe307208'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:46.822481 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9d99cbc5-bd95-4f8e-b3c2-ef3efe307208 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d4480c2d4ed4078b9f51264aa2272a9', 'request_id': 'req-9d99cbc5-bd95-4f8e-b3c2-ef3efe307208'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:46.827925 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9d99cbc5-bd95-4f8e-b3c2-ef3efe307208 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d4480c2d4ed4078b9f51264aa2272a9', 'request_id': 'req-9d99cbc5-bd95-4f8e-b3c2-ef3efe307208'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:46.838763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9d99cbc5-bd95-4f8e-b3c2-ef3efe307208 admin admin] The token cache is being invalidate because project 9d4480c2d4ed4078b9f51264aa2272a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:13:46.839045 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9d99cbc5-bd95-4f8e-b3c2-ef3efe307208 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:46.839673 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-9d99cbc5-bd95-4f8e-b3c2-ef3efe307208 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:13:46.841213 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1433/2909] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:13:46 2022] DELETE /identity/v3/projects/9d4480c2d4ed4078b9f51264aa2272a9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:13:46.846083 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426dda7-b692-468b-b644-f2732cdcffde None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:46.846377 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426dda7-b692-468b-b644-f2732cdcffde None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:46.846661 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426dda7-b692-468b-b644-f2732cdcffde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:46.865209 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-b426dda7-b692-468b-b644-f2732cdcffde None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:13:46.866422 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-b426dda7-b692-468b-b644-f2732cdcffde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:46.925441 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1477/2910] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:13:46.935038 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0419b7b-9ec0-4f90-b44f-799c5b632593 None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:46.935261 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0419b7b-9ec0-4f90-b44f-799c5b632593 None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:46.935556 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0419b7b-9ec0-4f90-b44f-799c5b632593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:46.955280 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-c0419b7b-9ec0-4f90-b44f-799c5b632593 None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:13:46.956493 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-c0419b7b-9ec0-4f90-b44f-799c5b632593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:46.964544 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1434/2911] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:13:46.968816 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426dda7-b692-468b-b644-f2732cdcffde None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:46.979000 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426dda7-b692-468b-b644-f2732cdcffde None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:46.983564 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d9f57-293a-4867-ba76-9276c0bedc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:46.984244 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d9f57-293a-4867-ba76-9276c0bedc91 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:46.984527 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d9f57-293a-4867-ba76-9276c0bedc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:46.984805 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d9f57-293a-4867-ba76-9276c0bedc91 admin admin] PATH_INFO: `/v3/domains` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:46.985530 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d9f57-293a-4867-ba76-9276c0bedc91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87061) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 21:13:46.985837 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d9f57-293a-4867-ba76-9276c0bedc91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:46.987427 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d9f57-293a-4867-ba76-9276c0bedc91 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:46.999605 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1478/2912] 10.222.0.60 () {58 vars in 1152 bytes} [Thu Jul 21 21:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 21:13:47.003766 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0419b7b-9ec0-4f90-b44f-799c5b632593 None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:47.012866 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0419b7b-9ec0-4f90-b44f-799c5b632593 None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:47.017683 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf912d10-167e-4186-b76a-b96273f8bfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:47.018259 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf912d10-167e-4186-b76a-b96273f8bfa8 admin admin] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:47.018565 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf912d10-167e-4186-b76a-b96273f8bfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:47.018796 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf912d10-167e-4186-b76a-b96273f8bfa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:47.019537 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf912d10-167e-4186-b76a-b96273f8bfa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:47.021619 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf912d10-167e-4186-b76a-b96273f8bfa8 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:47.046606 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1435/2913] 10.222.0.60 () {60 vars in 1152 bytes} [Thu Jul 21 21:13:47 2022] POST /identity/v3/projects => generated 375 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 21:13:47.051433 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d9f57-293a-4867-ba76-9276c0bedc91 admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:47.061427 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d9f57-293a-4867-ba76-9276c0bedc91 admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:47.117520 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d85e8f-bad8-4ba0-ac9a-088c2c2e296c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:47.118210 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d85e8f-bad8-4ba0-ac9a-088c2c2e296c admin admin] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:47.118491 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d85e8f-bad8-4ba0-ac9a-088c2c2e296c admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:47.118763 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d85e8f-bad8-4ba0-ac9a-088c2c2e296c admin admin] PATH_INFO: `/v3/users` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:47.119458 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d85e8f-bad8-4ba0-ac9a-088c2c2e296c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:47.121695 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d85e8f-bad8-4ba0-ac9a-088c2c2e296c admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:47.142390 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1479/2914] 10.222.0.60 () {60 vars in 1143 bytes} [Thu Jul 21 21:13:47 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 21:13:47.148266 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf912d10-167e-4186-b76a-b96273f8bfa8 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:47.158477 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf912d10-167e-4186-b76a-b96273f8bfa8 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:47.163519 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e043e-58ef-4745-a23a-3bdc6dbbefe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:47.164215 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e043e-58ef-4745-a23a-3bdc6dbbefe7 admin admin] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:47.164498 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e043e-58ef-4745-a23a-3bdc6dbbefe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:47.164768 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e043e-58ef-4745-a23a-3bdc6dbbefe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:47.165516 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e043e-58ef-4745-a23a-3bdc6dbbefe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:47.167094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e043e-58ef-4745-a23a-3bdc6dbbefe7 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:47.179431 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1436/2915] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 21:13:47.183382 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d85e8f-bad8-4ba0-ac9a-088c2c2e296c admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:47.193936 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d85e8f-bad8-4ba0-ac9a-088c2c2e296c admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:47.198421 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0afb9e-c884-4566-be69-675c509ba005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:47.199077 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0afb9e-c884-4566-be69-675c509ba005 admin admin] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:47.199361 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0afb9e-c884-4566-be69-675c509ba005 admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:47.199635 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0afb9e-c884-4566-be69-675c509ba005 admin admin] PATH_INFO: `/v3/roles` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:47.200391 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0afb9e-c884-4566-be69-675c509ba005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:47.201959 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0afb9e-c884-4566-be69-675c509ba005 admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:47.211945 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1480/2916] 10.222.0.60 () {58 vars in 1121 bytes} [Thu Jul 21 21:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 21:13:47.216115 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e043e-58ef-4745-a23a-3bdc6dbbefe7 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:47.226736 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e043e-58ef-4745-a23a-3bdc6dbbefe7 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:47.231345 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b64cb-287e-41f0-af72-0b4729496a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:47.232246 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b64cb-287e-41f0-af72-0b4729496a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:47.232530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b64cb-287e-41f0-af72-0b4729496a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:47.232801 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b64cb-287e-41f0-af72-0b4729496a42 admin admin] PATH_INFO: `/v3/projects/af3f544fda5c42b8b7c168f798d5d553/users/09c67ee6a6db43d885c8f52a8445bd56/roles/22f9190503114c4aba5e898e7a460546` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:47.248524 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b64cb-287e-41f0-af72-0b4729496a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af3f544fda5c42b8b7c168f798d5d553, user_id=09c67ee6a6db43d885c8f52a8445bd56, role_id=22f9190503114c4aba5e898e7a460546)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:47.251192 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b64cb-287e-41f0-af72-0b4729496a42 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:47.265167 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1437/2917] 10.222.0.60 () {60 vars in 1482 bytes} [Thu Jul 21 21:13:47 2022] PUT /identity/v3/projects/af3f544fda5c42b8b7c168f798d5d553/users/09c67ee6a6db43d885c8f52a8445bd56/roles/22f9190503114c4aba5e898e7a460546 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:13:47.274769 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6d025-2f23-4b94-8543-97c367ef4296 None None] REQUEST_METHOD: `POST` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:47.275206 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6d025-2f23-4b94-8543-97c367ef4296 None None] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:47.275406 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6d025-2f23-4b94-8543-97c367ef4296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:47.302583 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.auth.core [None req-3fb6d025-2f23-4b94-8543-97c367ef4296 None None] MFA Rules not processed for user `09c67ee6a6db43d885c8f52a8445bd56`. Rule list: `[]` (Enabled: `True`). {{(pid=87061) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:13:47.303820 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-3fb6d025-2f23-4b94-8543-97c367ef4296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:47.336086 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:13:47.370599 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1481/2918] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:13:47.383412 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b64cb-287e-41f0-af72-0b4729496a42 admin admin] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:13:47.385109 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-b77b64cb-287e-41f0-af72-0b4729496a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:13:47.431156 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:13:47.467590 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b64cb-287e-41f0-af72-0b4729496a42 admin admin] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:13:47.471621 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33f6c8-5881-4ac1-bb5c-2158343515ee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d596de177e24ca5a254bda6644a07da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:13:47.473094 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33f6c8-5881-4ac1-bb5c-2158343515ee service glance] REQUEST_METHOD: `GET` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:13:47.473552 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33f6c8-5881-4ac1-bb5c-2158343515ee service glance] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:13:47.474012 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33f6c8-5881-4ac1-bb5c-2158343515ee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:13:47.483272 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33f6c8-5881-4ac1-bb5c-2158343515ee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:13:47.484711 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33f6c8-5881-4ac1-bb5c-2158343515ee service glance] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:13:47.493599 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1438/2919] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:13:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:13:57.367770 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=12 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:13:57.489610 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=16 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:14:10.284138 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6d025-2f23-4b94-8543-97c367ef4296 None None] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:14:10.345668 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6d025-2f23-4b94-8543-97c367ef4296 None None] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:14:10.350153 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1feac-c719-463d-84a3-86b0980e56bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:14:10.351036 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1feac-c719-463d-84a3-86b0980e56bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:14:10.351321 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1feac-c719-463d-84a3-86b0980e56bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:14:10.351600 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1feac-c719-463d-84a3-86b0980e56bb admin admin] PATH_INFO: `/v3/users/09c67ee6a6db43d885c8f52a8445bd56` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:14:10.352893 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1feac-c719-463d-84a3-86b0980e56bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c67ee6a6db43d885c8f52a8445bd56)` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:14:10.354793 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1feac-c719-463d-84a3-86b0980e56bb admin admin] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:14:10.401647 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7df1feac-c719-463d-84a3-86b0980e56bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c67ee6a6db43d885c8f52a8445bd56'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:14:10.402309 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7df1feac-c719-463d-84a3-86b0980e56bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c67ee6a6db43d885c8f52a8445bd56'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:14:10.415864 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7df1feac-c719-463d-84a3-86b0980e56bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c67ee6a6db43d885c8f52a8445bd56'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:14:10.423322 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7df1feac-c719-463d-84a3-86b0980e56bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c67ee6a6db43d885c8f52a8445bd56'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:14:10.430831 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.notifications [None req-7df1feac-c719-463d-84a3-86b0980e56bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c67ee6a6db43d885c8f52a8445bd56'} {{(pid=87061) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:14:10.436823 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1482/2920] 10.222.0.60 () {58 vars in 1223 bytes} [Thu Jul 21 21:14:10 2022] DELETE /identity/v3/users/09c67ee6a6db43d885c8f52a8445bd56 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:14:10.441410 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877e5e2-18f3-4252-ae65-ba3050fa0edf None None] REQUEST_METHOD: `POST` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:14:10.447059 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877e5e2-18f3-4252-ae65-ba3050fa0edf None None] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:14:10.447059 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877e5e2-18f3-4252-ae65-ba3050fa0edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:14:10.459491 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.auth.core [None req-6877e5e2-18f3-4252-ae65-ba3050fa0edf None None] MFA Rules not processed for user `5ca745bbd8294b9eaae8eb567675a772`. Rule list: `[]` (Enabled: `True`). {{(pid=87060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 21:14:10.461447 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6877e5e2-18f3-4252-ae65-ba3050fa0edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:14:10.520306 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1439/2921] 10.222.0.60 () {58 vars in 957 bytes} [Thu Jul 21 21:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 21:14:10.529884 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1feac-c719-463d-84a3-86b0980e56bb admin admin] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:14:10.531327 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-7df1feac-c719-463d-84a3-86b0980e56bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:14:10.587729 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1feac-c719-463d-84a3-86b0980e56bb admin admin] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:14:10.591300 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6418204-500e-4e40-abb1-3bd898f6078b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aff48933844401b86eb808937e35c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:14:10.592003 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6418204-500e-4e40-abb1-3bd898f6078b service neutron] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:14:10.592269 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6418204-500e-4e40-abb1-3bd898f6078b service neutron] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:14:10.592543 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6418204-500e-4e40-abb1-3bd898f6078b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:14:10.601662 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6418204-500e-4e40-abb1-3bd898f6078b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:14:10.603073 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6418204-500e-4e40-abb1-3bd898f6078b service neutron] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:14:10.612566 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1483/2922] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:14:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:14:10.875083 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877e5e2-18f3-4252-ae65-ba3050fa0edf None None] Authenticating user token {{(pid=87060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:14:10.876574 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.fernet_utils [None req-6877e5e2-18f3-4252-ae65-ba3050fa0edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:14:10.888856 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877e5e2-18f3-4252-ae65-ba3050fa0edf None None] Validating token access rules against request {{(pid=87060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:14:10.893625 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58333b-81ec-4398-9b64-9b71ab3c9b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ca745bbd8294b9eaae8eb567675a772', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9f8292e70024a6985791def05952106', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:14:10.894488 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58333b-81ec-4398-9b64-9b71ab3c9b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:14:10.894763 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58333b-81ec-4398-9b64-9b71ab3c9b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:14:10.895032 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58333b-81ec-4398-9b64-9b71ab3c9b79 admin admin] PATH_INFO: `/v3/projects/af3f544fda5c42b8b7c168f798d5d553` {{(pid=87060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:14:10.896235 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58333b-81ec-4398-9b64-9b71ab3c9b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af3f544fda5c42b8b7c168f798d5d553)` {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:14:10.898819 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58333b-81ec-4398-9b64-9b71ab3c9b79 admin admin] RBAC: Authorization granted {{(pid=87060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:14:10.945908 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-7f58333b-81ec-4398-9b64-9b71ab3c9b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af3f544fda5c42b8b7c168f798d5d553', 'request_id': 'req-7f58333b-81ec-4398-9b64-9b71ab3c9b79'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:14:10.953651 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-7f58333b-81ec-4398-9b64-9b71ab3c9b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af3f544fda5c42b8b7c168f798d5d553', 'request_id': 'req-7f58333b-81ec-4398-9b64-9b71ab3c9b79'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:14:10.959357 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-7f58333b-81ec-4398-9b64-9b71ab3c9b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af3f544fda5c42b8b7c168f798d5d553', 'request_id': 'req-7f58333b-81ec-4398-9b64-9b71ab3c9b79'} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:14:10.971016 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-7f58333b-81ec-4398-9b64-9b71ab3c9b79 admin admin] The token cache is being invalidate because project af3f544fda5c42b8b7c168f798d5d553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87060) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 21:14:10.971304 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-7f58333b-81ec-4398-9b64-9b71ab3c9b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:14:10.971970 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG keystone.notifications [None req-7f58333b-81ec-4398-9b64-9b71ab3c9b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87060) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 21:14:10.973530 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: [pid: 87060|app: 0|req: 1440/2923] 10.222.0.60 () {58 vars in 1232 bytes} [Thu Jul 21 21:14:10 2022] DELETE /identity/v3/projects/af3f544fda5c42b8b7c168f798d5d553 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 21:14:20.608356 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:14:20.968041 dv-cinder-850687-1-iscsi devstack@keystone.service[87060]: DEBUG dbcounter [-] [87060] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87060) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 21:16:35.910560 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6418204-500e-4e40-abb1-3bd898f6078b service neutron] Authenticating user token {{(pid=87061) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 21:16:35.911999 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-a6418204-500e-4e40-abb1-3bd898f6078b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:16:35.978797 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6418204-500e-4e40-abb1-3bd898f6078b service neutron] Validating token access rules against request {{(pid=87061) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 21:16:36.036324 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe56700-0558-4fd6-b43c-f0c1345cceb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4786051321f4a6f80a6cd23eacb09bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6bc61fe15894891a9054911321f4ac3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87061) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 21:16:36.036965 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe56700-0558-4fd6-b43c-f0c1345cceb2 service placement] REQUEST_METHOD: `GET` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 21:16:36.037246 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe56700-0558-4fd6-b43c-f0c1345cceb2 service placement] SCRIPT_NAME: `/identity` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 21:16:36.037519 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe56700-0558-4fd6-b43c-f0c1345cceb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87061) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 21:16:36.039271 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.fernet_utils [None req-ebe56700-0558-4fd6-b43c-f0c1345cceb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87061) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 21:16:36.050328 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe56700-0558-4fd6-b43c-f0c1345cceb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 21:16:36.051903 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe56700-0558-4fd6-b43c-f0c1345cceb2 service placement] RBAC: Authorization granted {{(pid=87061) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 21:16:36.061552 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: [pid: 87061|app: 0|req: 1484/2924] 10.222.0.60 () {60 vars in 1365 bytes} [Thu Jul 21 21:16:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 21:16:46.057402 dv-cinder-850687-1-iscsi devstack@keystone.service[87061]: DEBUG dbcounter [-] [87061] Writing DB stats keystone:SELECT=58 {{(pid=87061) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}